Advertisements

Cognitive Services : Convert Text to Speech in multiple languages using Asp.Net Core & C#


Introduction

In this article, we are going to learn how to convert text to speech in multiple languages using one of the important Cognitive Services API called Microsoft Text to Speech Service API ( One of the API in Speech API ). The Text to Speech (TTS) API of the Speech service converts input text into natural-sounding speech (also called as speech synthesis). It supports text in multiple languages  and gender based voice(male or female)

You can also refer the following articles on Cognitive Service.

Prerequisites

  1. Subscription key ( Azure Portal ) or Trail Subscription Key
  2. Visual Studio 2015 or 2017

Convert Text to Speech API

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Speech Service API in the Azure portal.
So please click on the “Create a resource” on the left top menu and search “Speech” in the search bar on the right side window or top of Azure Marketplace.

Now we can see there are few speech related “AI + Machine Learning ” categories listed in the search result.

Click on the “create” button to create Speech Service API.

Provision a Speech Service API ( Text to Speech ) Subscription Key

After clicking the “Create”, It will open another window. There we need to provide the basic information about Speech API.

Name : Name of the Translator Text API ( Eg. TextToSpeechApp ).

Subscription : We can select our Azure subscription for Speech API creation.

Location : We can select  location of resource group. The best thing is we can choose a location closest to our customer.

Pricing tier : Select an appropriate pricing tier for our requirement.

Resource group : We can create a new resource group or choose from an existing one ( We created a new resource group as “SpeechResource” ).

Now click on the “TextToSpeechApp” in dashboard page and it will redirect to the detailed page of TextToSpeechApp ( “Overview” ). Here, we can see the “Keys” ( Subscription key details ) menu in the left side panel. Then click on the “Keys” menu and it will open the Subscription Keys details. We can use any of the subscription keys or regenerate the given key for text to speech conversion using Microsoft Speech Service API.

Authentication

A token ( bearer ) based authentication is required in the Text To Speech conversion using Speech Service API. So we need to create an authentication token using “TextToSpeechApp” subscription keys. The following “endPoint” will help to create an authentication token for Text to speech conversion. The each access token is valid for 10 minutes and after that we need to create a new one for the next process.

https://westus.api.cognitive.microsoft.com/sts/v1.0/issueToken”

Speech Synthesis Markup Language ( SSML )

The Speech Synthesis Markup Language (SSML) is an XML-based markup language that provides a way to control the pronunciation and rhythm of text-to-speech. More about SSML ..

SSML Format :

<speak version='1.0' xml:lang='en-US'><voice xml:lang='ta-IN' xml:gender='Female' name='Microsoft Server Speech Text to Speech Voice (ta-IN, Valluvar)'>
        நன்றி
</voice></speak>

How to make a request

This is very simple process,  HTTP request is made in POST method. So that means we need to pass secure data in the request body and that will be a plain text or a SSML document. As per the documentation,it is clearly mentioned in most cases that we need to use SSML body as request. The maximum length of the HTTP request body is 1024 characters and the following is the endPoint for our http Post method.

https://westus.tts.speech.microsoft.com/cognitiveservices/v1&#8221;

The following are the HTTP headers required in the request body.

Pic source : https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-text-to-speech

Index.html

The following html contains the binding methodology that we have used in our application by using the  latest Tag helpers of ASP.Net Core.

Model

The following model contains the Speech Model information.

using Microsoft.AspNetCore.Mvc.Rendering;
using System.Collections.Generic;
using System.ComponentModel;

namespace TextToSpeechApp.Models
{
    public class SpeechModel
    {
        public string Content { get; set; }

        public string SubscriptionKey { get; set; } = "< Subscription Key >";

        [DisplayName("Language Selection :")]
        public string LanguageCode { get; set; } = "NA";

        public List<SelectListItem> LanguagePreference { get; set; } = new List<SelectListItem>
        {
        new SelectListItem { Value = "NA", Text = "-Select-" },
        new SelectListItem { Value = "en-US", Text = "English (United States)"  },
        new SelectListItem { Value = "en-IN", Text = "English (India)"  },
        new SelectListItem { Value = "ta-IN", Text = "Tamil (India)"  },
        new SelectListItem { Value = "hi-IN", Text = "Hindi (India)"  },
        new SelectListItem { Value = "te-IN", Text = "Telugu (India)"  }
        };
    }
}

Interface

The “ITextToSpeech” contains one signature for converting text to speech based on the given input. So we have injected this interface in the ASP.NET Core “Startup.cs” class as a “AddTransient”.

using System.Threading.Tasks;

namespace TextToSpeechApp.BusinessLayer.Interface
{
    public interface ITextToSpeech
    {
        Task<byte[]> TranslateText(string token, string key, string content, string lang);
    }
}

Text to Speech API Service

We can add the valid Speech API Subscription key and authentication token into the following code.

/// 

<summary>
        /// Translate text to speech
        /// </summary>


        /// <param name="token">Authentication token</param>
        /// <param name="key">Azure subscription key</param>
        /// <param name="content">Text content for speech</param>
        /// <param name="lang">Speech conversion language</param>
        /// <returns></returns>
        public async Task<byte[]> TranslateText(string token, string key, string content, string lang)
        {
            //Request url for the speech api.
            string uri = "https://westus.tts.speech.microsoft.com/cognitiveservices/v1";
            //Generate Speech Synthesis Markup Language (SSML) 
            var requestBody = this.GenerateSsml(lang, "Female", this.ServiceName(lang), content);

            using (var client = new HttpClient())
            using (var request = new HttpRequestMessage())
            {
                request.Method = HttpMethod.Post;
                request.RequestUri = new Uri(uri);
                request.Headers.Add("Ocp-Apim-Subscription-Key", key);
                request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
                request.Headers.Add("X-Microsoft-OutputFormat", "audio-16khz-64kbitrate-mono-mp3");
                request.Content = new StringContent(requestBody, Encoding.UTF8, "text/plain");
                request.Content.Headers.Remove("Content-Type");
                request.Content.Headers.Add("Content-Type", "application/ssml+xml");
                request.Headers.Add("User-Agent", "TexttoSpeech");
                var response = await client.SendAsync(request);
                var httpStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false);
                Stream receiveStream = httpStream;
                byte[] buffer = new byte[32768];

                using (Stream stream = httpStream)
                {
                    using (MemoryStream ms = new MemoryStream())
                    {
                        byte[] waveBytes = null;
                        int count = 0;
                        do
                        {
                            byte[] buf = new byte[1024];
                            count = stream.Read(buf, 0, 1024);
                            ms.Write(buf, 0, count);
                        } while (stream.CanRead && count > 0);

                        waveBytes = ms.ToArray();

                        return waveBytes;
                    }
                }
            }
        }

Download

Demo

Output

The given text is converted into speech in desired  language listed in a drop-down list using Microsoft Speech API.

Reference

See Also

You can download other source codes from MSDN Code, using the link, mentioned below.

Summary

From this article we have learned how to convert text to speech in multiple languages using Asp.Net Core & C# as per the API documentation using one of the important Cognitive Services API ( Text to Speech API is a part of Speech API ). I hope this article is useful for all Azure Cognitive Services API beginners.

Advertisements

Cognitive Services : Translate Text into multiple languages using Asp.Net Core & C#


Introduction

In this article, we are going to learn how to translate text into multiple languages using one of the important Cognitive Services API called Microsoft Translate Text API ( One of the API in Language API ). It’s a simple cloud-based machine translation service and obviously we can test through simple Rest API call. Microsoft is using a new standard for high-quality AI-powered machine translations known as Neural Machine Translation (NMT).

Pic source : https://www.microsoft.com/en-us/translator/business/machine-translation/#whatmachine

You can also refer the following articles on Cognitive Service.

Prerequisites

  1. Subscription key ( Azure Portal ).
  2. Visual Studio 2015 or 2017

Translator Text API

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Translator Text API in the Azure portal. So please click on the “Create a resource” on the left top menu and search “Translator Text” in the search bar on the right side window or top of Azure Marketplace.

Click on the “create” button to create Translator Text API.

Provision a Translator Text Subscription Key

After clicking the “Create”, It will open another window. There we need to provide the basic information about Translator Text API.

Name : Name of the Translator Text API ( Eg. TranslatorTextApp ).

Subscription : We can select our Azure subscription for Translator Text  API creation.

Location : We can select our location of resource group. The best thing is we can choose a location closest to our customer.

Pricing tier : Select an appropriate pricing tier for our requirement.

Resource group : We can create a new resource group or choose from an existing one.

Now click on the “TranslatorTextApp” in dashboard page and it will redirect to the detailed page of  TranslatorTextApp ( “Overview” ). Here, we can see the “Keys” ( Subscription key details ) menu in the left side panel. Then click on the “Keys” menu and it will open the Subscription Keys details. We can use any of the subscription keys or regenerate the given key for text translation using Microsoft Translator Text API.

Language Request URL

The following request url gets the set of languages currently supported by other operations of the Microsoft Translator Text API.

https://api.cognitive.microsofttranslator.com/languages?api-version=3.0

Endpoint

The version of the API requested by the client and the Value must be 3.0 and also we can include query parameters and request header in the following endPoint used in our application.

https://api.cognitive.microsofttranslator.com/translate?api-version=3.0

Mandatory required parameters in the query string are “api-version” and “to” . The “api-version” value must be “3.0” as per the current documentation. “to” is the language code parameter used for translating the entered text into the desired language.

The mandatory request headers are “authorization header” and “Content-Type”. We can pass our subscription key into the “authorization header” and the simplest way is to pass our Azure secret key to the Translator service using request header “Ocp-Apim-Subscription-Key”.

 Index.html

The following html contains the binding methodology that we have used in our application by using the  latest Tag helpers of ASP.Net Core.

site.js

The following ajax call will trigger for each drop-down index change in the language selection using drop-down list.

// Write your JavaScript code.
$(function () {
    $(document)
        .on('change', '#ddlLangCode', function () {
            var languageCode = $(this).val();
            var enterText = $("#enterText").val();
            if (1 <= $("#enterText").val().trim().length && languageCode != "NA") {

                $('#enterText').removeClass('redBorder');

                var url = '/Home/Index';
                var dataToSend = { "LanguageCode": languageCode, "Text": enterText };
                dataType: "json",
                    $.ajax({
                        url: url,
                        data: dataToSend,
                        type: 'POST',
                        success: function (response) {
                            //update control on View
                            var result = JSON.parse(response);
                            var translatedText = result[0].translations[0].text;
                            $('#translatedText').val(translatedText);
                        }
                    })
            }
            else {
                $('#enterText').addClass('redBorder');
                $('#translatedText').val("");
            }
        });
});

Interface

The “ITranslateText” contains one signature for translating text content based on the given input. So we have injected this interface in the ASP.NET Core “Startup.cs” class as a “AddTransient”.

using System.Threading.Tasks;

namespace TranslateTextApp.Business_Layer.Interface
{
    interface ITranslateText
    {
        Task<string> Translate(string uri, string text, string key);
    }
}

Translator Text API Service

We can add the valid Translator Text API Subscription Key into the following code.

using Newtonsoft.Json;
using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using TranslateTextApp.Business_Layer.Interface;

namespace TranslateTextApp.Business_Layer
{
    public class TranslateTextService : ITranslateText
    {
        /// 
<summary>
        /// Translate the given text in to selected language.
        /// </summary>

        /// <param name="uri">Request uri</param>
        /// <param name="text">The text is given for translation</param>
        /// <param name="key">Subscription key</param>
        /// <returns></returns>
        public async Task<string> Translate(string uri, string text, string key)
        {
            System.Object[] body = new System.Object[] { new { Text = text } };
            var requestBody = JsonConvert.SerializeObject(body);
            
            using (var client = new HttpClient())
            using (var request = new HttpRequestMessage())
            {
                request.Method = HttpMethod.Post;
                request.RequestUri = new Uri(uri);
                request.Content = new StringContent(requestBody, Encoding.UTF8, "application/json");
                request.Headers.Add("Ocp-Apim-Subscription-Key", key);

                var response = await client.SendAsync(request);
                var responseBody = await response.Content.ReadAsStringAsync();
                dynamic result = JsonConvert.SerializeObject(JsonConvert.DeserializeObject(responseBody), Formatting.Indented);
                
                return result;
            }
        }
    }
}

API Response – Based on the given text

The successful json response.

[
  {
    "detectedLanguage": {
      "language": "en",
      "score": 1.0
    },
    "translations": [
      {
        "text": "सफलता का कोई शार्टकट नहीं होता",
        "to": "hi"
      }
    ]
  }
]

Download

Output

The given text is translated into desired  language listed in a drop-down list using Microsoft Translator API.

 

Reference

See Also

You can download other source codes from MSDN Code, using the link, mentioned below.

Summary

From this article we have learned translate a text(typed in english) in to different languages as per the API documentation using one of the important Cognitive Services API ( Translator Text API is a part of Language API ). I hope this article is useful for all Azure Cognitive Services API beginners.

Cognitive Services – Optical Character Recognition (OCR) from an image using Computer Vision API And C#


Introduction

In our previous article we learned how to Analyze an Image Using Computer Vision API With ASP.Net Core & C#. In this article we are going to learn how to extract printed text also known as optical character recognition (OCR) from an image using one of the important Cognitive Services API called as Computer Vision API. So we need a valid subscription key for accessing this feature in an image.

Optical Character Recognition (OCR)

Optical Character Recognition (OCR) detects text in an image and extracts the recognized characters into a machine-usable character stream.

Prerequisites

  1. Subscription key ( Azure Portal ).
  2. Visual Studio 2015 or 2017

Subscription Key Free Trail

If you don’t have Microsoft Azure Subscription and want to test the Computer Vision API because it requires a valid Subscription key for processing the image information. Don’t worry !! Microsoft gives a 7 day trial Subscription Key ( Click here ). We can use that Subscription key for testing purposes. If you sign up using the Computer Vision free trial, then your subscription keys are valid for the westcentral region (https://westcentralus.api.cognitive.microsoft.com ).

Requirements

These are the major requirements mentioned in the Microsoft docs.

  1. Supported input methods: Raw image binary in the form of an application/octet stream or image URL.
  2. Supported image formats: JPEG, PNG, GIF, BMP.
  3. Image file size: Less than 4 MB.
  4. Image dimension: Greater than 50 x 50 pixels.

Computer Vision API

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Computer Vision Subscription Key in the Azure portal.

Click on “Create a resource” on the left side menu and it will open an “Azure Marketplace”. There, we can see the list of services. Click “AI + Machine Learning” then click on the “Computer Vision”.

Provision a Computer Vision Subscription Key

After clicking the “Computer Vision”, It will open another section. There, we need to provide the basic information about Computer Vision API.

Name : Name of the Computer Vision API ( Eg. OCRApp ).

Subscription : We can select our Azure subscription for Computer Vision API creation.

Location : We can select our location of resource group. The best thing is we can choose a location closest to our customer.

Pricing tier : Select an appropriate pricing tier for our requirement.

Resource group : We can create a new resource group or choose from an existing one.

Now click on the “OCRApp” in dashboard page and it will redirect to the details page of OCRApp ( “Overview” ). Here, we can see the Manage Key ( Subscription key details ) & Endpoint details. Click on the Show access keys links and it will redirect to another page.

We can use any of the subscription keys or regenerate the given key for getting image information using Computer Vision API.

 

Endpoint

As we mentioned above the location is the same for all the free trial Subscription Keys. In Azure we can choose available locations while creating a Computer Vision API. We have used the following endpoint in our code.

https://westus.api.cognitive.microsoft.com/vision/v1.0/ocr

View Model

The following model will contain the API image response information.

using System.Collections.Generic;

namespace OCRApp.Models
{
    public class Word
    {
        public string boundingBox { get; set; }
        public string text { get; set; }
    }

    public class Line
    {
        public string boundingBox { get; set; }
        public List<Word> words { get; set; }
    }

    public class Region
    {
        public string boundingBox { get; set; }
        public List<Line> lines { get; set; }
    }

    public class ImageInfoViewModel
    {
        public string language { get; set; }
        public string orientation { get; set; }
        public int textAngle { get; set; }
        public List<Region> regions { get; set; }
    }
}

Request URL

We can add additional parameters or request parameters ( optional ) in our API “endPoint” and it will provide more information for the given image.

https://[location].api.cognitive.microsoft.com/vision/v1.0/ocr[?language][&detectOrientation ]

Request parameters

These are the following optional parameters available in computer vision API.

  1. language
  2. detectOrientation

language

The service will detect 26 languages of the text in the image and It will contain “unk” as the default value. That means the service will auto detect the language of the text in the image.

The following are the supported language mention in the Microsoft API documentation.

  1. unk (AutoDetect)
  2. en (English)
  3. zh-Hans (ChineseSimplified)
  4. zh-Hant (ChineseTraditional)
  5. cs (Czech)
  6. da (Danish)
  7. nl (Dutch)
  8. fi (Finnish)
  9. fr (French)
  10. de (German)
  11. el (Greek)
  12. hu (Hungarian)
  13. it (Italian)
  14. ja (Japanese)
  15. ko (Korean)
  16. nb (Norwegian)
  17. pl (Polish)
  18. pt (Portuguese,
  19. ru (Russian)
  20. es (Spanish)
  21. sv (Swedish)
  22. tr (Turkish)
  23. ar (Arabic)
  24. ro (Romanian)
  25. sr-Cyrl (SerbianCyrillic)
  26. sr-Latn (SerbianLatin)
  27. sk (Slovak)

detectOrientation

This will detect the text orientation in the image, for this feature we need to add detectOrientation=true in the service url  or Request url as we discussed earlier.

Vision API Service

The following code will process and generate image information using Computer Vision API and its response is mapped into the “ImageInfoViewModel”. We can add the valid Computer Vision API Subscription Key into the following code.

using Newtonsoft.Json;
using OCRApp.Models;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;

namespace OCRApp.Business_Layer
{
    public class VisionApiService
    {
        // Replace <Subscription Key> with your valid subscription key.
        const string subscriptionKey = "<Subscription Key>";

        // You must use the same region in your REST call as you used to
        // get your subscription keys. The paid subscription keys you will get
        // it from microsoft azure portal.
        // Free trial subscription keys are generated in the westcentralus region.
        // If you use a free trial subscription key, you shouldn't need to change
        // this region.
        const string endPoint =
            "https://westus.api.cognitive.microsoft.com/vision/v1.0/ocr";

        /// 
<summary>
        /// Gets the text visible in the specified image file by using
        /// the Computer Vision REST API.
        /// </summary>

        public async Task<string> MakeOCRRequest()
        {
            string imageFilePath = @"C:\Users\rajeesh.raveendran\Desktop\bill.jpg";
            var errors = new List<string>();
            string extractedResult = "";
            ImageInfoViewModel responeData = new ImageInfoViewModel();

            try
            {
                HttpClient client = new HttpClient();

                // Request headers.
                client.DefaultRequestHeaders.Add(
                    "Ocp-Apim-Subscription-Key", subscriptionKey);

                // Request parameters.
                string requestParameters = "language=unk&detectOrientation=true";

                // Assemble the URI for the REST API Call.
                string uri = endPoint + "?" + requestParameters;

                HttpResponseMessage response;

                // Request body. Posts a locally stored JPEG image.
                byte[] byteData = GetImageAsByteArray(imageFilePath);

                using (ByteArrayContent content = new ByteArrayContent(byteData))
                {
                    // This example uses content type "application/octet-stream".
                    // The other content types you can use are "application/json"
                    // and "multipart/form-data".
                    content.Headers.ContentType =
                        new MediaTypeHeaderValue("application/octet-stream");

                    // Make the REST API call.
                    response = await client.PostAsync(uri, content);
                }

                // Get the JSON response.
                string result = await response.Content.ReadAsStringAsync();

                //If it is success it will execute further process.
                if (response.IsSuccessStatusCode)
                {
                    // The JSON response mapped into respective view model.
                    responeData = JsonConvert.DeserializeObject<ImageInfoViewModel>(result,
                        new JsonSerializerSettings
                        {
                            NullValueHandling = NullValueHandling.Include,
                            Error = delegate (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs earg)
                            {
                                errors.Add(earg.ErrorContext.Member.ToString());
                                earg.ErrorContext.Handled = true;
                            }
                        }
                    );

                    var linesCount = responeData.regions[0].lines.Count;
                    for (int i = 0; i < linesCount; i++)
                    {
                        var wordsCount = responeData.regions[0].lines[i].words.Count;
                        for (int j = 0; j < wordsCount; j++)
                        {
                            //Appending all the lines content into one.
                            extractedResult += responeData.regions[0].lines[i].words[j].text + " ";
                        }
                        extractedResult += Environment.NewLine;
                    }

                }
            }
            catch (Exception e)
            {
                Console.WriteLine("\n" + e.Message);
            }
            return extractedResult;
        }

        /// 
<summary>
        /// Returns the contents of the specified file as a byte array.
        /// </summary>

        /// <param name="imageFilePath">The image file to read.</param>
        /// <returns>The byte array of the image data.</returns>
        static byte[] GetImageAsByteArray(string imageFilePath)
        {
            using (FileStream fileStream =
                new FileStream(imageFilePath, FileMode.Open, FileAccess.Read))
            {
                BinaryReader binaryReader = new BinaryReader(fileStream);
                return binaryReader.ReadBytes((int)fileStream.Length);
            }
        }
    }

}

API Response – Based on the given Image

The successful json response.

{
  "language": "en",
  "orientation": "Up",
  "textAngle": 0,
  "regions": [
    {
      "boundingBox": "306,69,292,206",
      "lines": [
        {
          "boundingBox": "306,69,292,24",
          "words": [
            {
              "boundingBox": "306,69,17,19",
              "text": "\"I"
            },
            {
              "boundingBox": "332,69,45,19",
              "text": "Will"
            },
            {
              "boundingBox": "385,69,88,24",
              "text": "Always"
            },
            {
              "boundingBox": "482,69,94,19",
              "text": "Choose"
            },
            {
              "boundingBox": "585,74,13,14",
              "text": "a"
            }
          ]
        },
        {
          "boundingBox": "329,100,246,24",
          "words": [
            {
              "boundingBox": "329,100,56,24",
              "text": "Lazy"
            },
            {
              "boundingBox": "394,100,85,19",
              "text": "Person"
            },
            {
              "boundingBox": "488,100,24,19",
              "text": "to"
            },
            {
              "boundingBox": "521,100,32,19",
              "text": "Do"
            },
            {
              "boundingBox": "562,105,13,14",
              "text": "a"
            }
          ]
        },
        {
          "boundingBox": "310,131,284,19",
          "words": [
            {
              "boundingBox": "310,131,95,19",
              "text": "Difficult"
            },
            {
              "boundingBox": "412,131,182,19",
              "text": "Job....Because"
            }
          ]
        },
        {
          "boundingBox": "326,162,252,24",
          "words": [
            {
              "boundingBox": "326,162,31,19",
              "text": "He"
            },
            {
              "boundingBox": "365,162,44,19",
              "text": "Will"
            },
            {
              "boundingBox": "420,162,52,19",
              "text": "Find"
            },
            {
              "boundingBox": "481,167,28,14",
              "text": "an"
            },
            {
              "boundingBox": "520,162,58,24",
              "text": "Easy"
            }
          ]
        },
        {
          "boundingBox": "366,193,170,24",
          "words": [
            {
              "boundingBox": "366,193,52,24",
              "text": "way"
            },
            {
              "boundingBox": "426,193,24,19",
              "text": "to"
            },
            {
              "boundingBox": "459,193,33,19",
              "text": "Do"
            },
            {
              "boundingBox": "501,193,35,19",
              "text": "It!\""
            }
          ]
        },
        {
          "boundingBox": "462,256,117,19",
          "words": [
            {
              "boundingBox": "462,256,37,19",
              "text": "Bill"
            },
            {
              "boundingBox": "509,256,70,19",
              "text": "Gates"
            }
          ]
        }
      ]
    }
  ]
}

Download

Output

Optical Character Recognition (OCR) from an image using Computer Vision API.

Reference

See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below.

Summary

From this article we have learned Optical Character Recognition (OCR) from an image using One of the important Cognitive Services API ( Computer Vision API ). I hope this article is useful for all Azure Cognitive Services API beginners.

 

Cognitive Services : Analyze an Image Using Computer Vision API With ASP.Net Core & C#


Introduction

One of the important Cognitive Services API is Computer Vision API and it helps to access the advanced algorithms for processing images and returning valuable information. For example By uploading an image or specifying an image URL, Microsoft Computer Vision algorithms can analyze visual content in different ways based on inputs and user choices. So we will get various information about the given image. We need a valid subscription key for accessing this feature.

Prerequisites

  1. Subscription key ( Azure Portal ).
  2. Visual Studio 2015 or 2017

Subscription Key Free Trail

If you don’t have Microsoft Azure Subscription and want to test the Computer Vision API because it requires a valid Subscription key for processing the image information. Don’t worry !! Microsoft gives a 7 day’s trail Subscription Key ( Click here ) . We can use that Subscription key for testing purpose. If you sign up using the Computer Vision free trial, Then your subscription keys are valid for the westcentral region ( https://westcentralus.api.cognitive.microsoft.com )

Requirements

These are the major requirements mention in the Microsoft docs.

  1. Supported input methods: Raw image binary in the form of an application/octet stream or image URL.
  2. Supported image formats: JPEG, PNG, GIF, BMP.
  3. Image file size: Less than 4 MB.
  4. Image dimension: Greater than 50 x 50 pixels.

Computer Vision API

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure Computer Vision Subscription Key in the Azure portal.

Click on “Create a resource” on the left side menu and it will open an “Azure Marketplace”. There, we can see the list of services. Click “AI + Machine Learning” then click on the “Computer Vision”.

Provision a Computer Vision Subscription Key

After clicking the “Computer Vision”, it will open another section. There, we need to provide the basic information about Computer Vision API.

Name : Name of the Computer Vision API.

Subscription : We can select our Azure subscription for Computer Vision API creation.

Location : We can select our location of resource group. The best thing is we can choose a location closest to our customer.

Pricing tier : Select an appropriate pricing tier for our requirement.

Resource group : We can create a new resource group or choose from an existing one.

Now click on the MenothVision in dashboard page and it will redirect to the details page of MenothVision ( “Overview” ). Here, we can see the Manage Key ( Subscription key details ) & Endpoint details. Click on the Show access keys links and it will redirect to another page.

We can use any of the Subscription key or Regenerate the given key for getting image information using Computer Vision API.

Endpoint

As we mentioned above the location is same for all the free trail Subscription Key. In Azure we can choose available locations while creating a Computer Vision API. The following Endpoint we have used in our code.

https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze

View Model

The following model will contain the API image response information.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace VisionApiDemo.Models
{
public class Detail
{
public List<object> celebrities { get; set; }
}

public class Category
{
public string name { get; set; }
public double score { get; set; }
public Detail detail { get; set; }
}

public class Caption
{
public string text { get; set; }
public double confidence { get; set; }
}

public class Description
{
public List<string> tags { get; set; }
public List<Caption> captions { get; set; }
}

public class Color
{
public string dominantColorForeground { get; set; }
public string dominantColorBackground { get; set; }
public List<string> dominantColors { get; set; }
public string accentColor { get; set; }
public bool isBwImg { get; set; }
}

public class Metadata
{
public int height { get; set; }
public int width { get; set; }
public string format { get; set; }
}

public class ImageInfoViewModel
{
public List<Category> categories { get; set; }
public Description description { get; set; }
public Color color { get; set; }
public string requestId { get; set; }
public Metadata metadata { get; set; }
}
}

Request URL

We can add additional parameters or request parameters ( optional ) in our API “endPoint” and it will provide more information for the given image.

https://%5Blocation%5D.api.cognitive.microsoft.com/vision/v1.0/analyze%5B?visualFeatures%5D%5B&details%5D%5B&language%5D

Request parameters

Currently we can use 3 optional parameters.

  1. visualFeatures
  2. details
  3. language

visualFeatures

The name itself clearly mentions it returns Visual Features of the given image. If we add multiple values in a visualFeatures parameters then put a comma for each value. The following are the visualFeatures parameters in API.

  • Categories
  • Tags
  • Description
  • Faces
  • ImageType
  • Color
  • Adult

details

This parameter will return domain specific information whether it is Celebrities or Landmarks.

Celebrities : If the detected image is of a celebrity it identify the same.

Landmarks : If the detected image is of a landmark it identify the same.

language

The service will return recognition results in specified language. Default language is english.

Supported languages.

  • en – English, Default.
  • zh – Simplified Chinese

Vision API Service

The following code will process and generate image information using Computer Vision API and its response is mapped into the “ImageInfoViewModel”. We can add the valid Computer Vision API Subscription Key into the following code.

using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using VisionApiDemo.Models;

namespace VisionApiDemo.Business_Layer
{
public class VisionApiService
{
const string subscriptionKey = "<Enter your subscriptionKey>";
const string endPoint =
"https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze";

public async Task<ImageInfoViewModel> MakeAnalysisRequest()
{
string imageFilePath = @"C:\Users\Rajeesh.raveendran\Desktop\Rajeesh.jpg";
var errors = new List<string>();
ImageInfoViewModel responeData = new ImageInfoViewModel();
try
{
HttpClient client = new HttpClient();

// Request headers.
client.DefaultRequestHeaders.Add(
"Ocp-Apim-Subscription-Key", subscriptionKey);

// Request parameters. A third optional parameter is "details".
string requestParameters =
"visualFeatures=Categories,Description,Color";

// Assemble the URI for the REST API Call.
string uri = endPoint + "?" + requestParameters;

HttpResponseMessage response;

// Request body. Posts a locally stored JPEG image.
byte[] byteData = GetImageAsByteArray(imageFilePath);

using (ByteArrayContent content = new ByteArrayContent(byteData))
{
// This example uses content type "application/octet-stream".
// The other content types you can use are "application/json"
// and "multipart/form-data".
content.Headers.ContentType =
new MediaTypeHeaderValue("application/octet-stream");

// Make the REST API call.
response = await client.PostAsync(uri, content);
}

// Get the JSON response.
var result = await response.Content.ReadAsStringAsync();

if (response.IsSuccessStatusCode)
{

responeData = JsonConvert.DeserializeObject<ImageInfoViewModel>(result,
new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Include,
Error = delegate (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs earg)
{
errors.Add(earg.ErrorContext.Member.ToString());
earg.ErrorContext.Handled = true;
}
}
);
}

}
catch (Exception e)
{
Console.WriteLine("\n" + e.Message);
}

return responeData;
}

static byte[] GetImageAsByteArray(string imageFilePath)
{
using (FileStream fileStream =
new FileStream(imageFilePath, FileMode.Open, FileAccess.Read))
{
BinaryReader binaryReader = new BinaryReader(fileStream);
return binaryReader.ReadBytes((int)fileStream.Length);
}
}
}
}

API Response – Based on the given Image

The successful json response.

{
"categories": [
{
"name": "people_group",
"score": 0.6171875,
"detail": {
"celebrities": []
}
},
{
"name": "people_many",
"score": 0.359375,
"detail": {
"celebrities": []
}
}
],
"description": {
"tags": [
"person",
"sitting",
"indoor",
"posing",
"group",
"people",
"man",
"photo",
"woman",
"child",
"front",
"young",
"table",
"cake",
"large",
"holding",
"standing",
"bench",
"room",
"blue"
],
"captions": [
{
"text": "a group of people sitting posing for the camera",
"confidence": 0.9833507086594954
}
]
},
"color": {
"dominantColorForeground": "White",
"dominantColorBackground": "White",
"dominantColors": [
"White",
"Black",
"Red"
],
"accentColor": "AD1E3E",
"isBwImg": false
},
"requestId": "89f21ccf-cb65-4107-8620-b920a03e5f03",
"metadata": {
"height": 346,
"width": 530,
"format": "Jpeg"
}
}

Download

Output

Image information captured using Computer Vision API.For demo purpose, I have taken only a few data even though you can get more information about the image.

Reference

See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below.

Summary

From this article we have learned how to implement One of the important Cognitive Services API ( Computer Vision API ). I hope this article is useful for all Azure Cognitive Services API beginners.

Create and Connect Azure SQL database in the Azure portal


Introduction

This article explains how to create and connect Azure SQL database in the Azure portal.

SQL database

First, we need to log into the Azure Portal with our Azure credentials. Then we need to create an Azure SQL database in the Azure portal.

Click on “Create a resource” on the left side menu and it will open an “Azure Marketplace”. There, we can see the list of services. Click “Databases” then click on the “SQL Database”.

SQL database

SQL database

Create a SQL database

After clicking the “SQL Database”, it will open another section. There, we need to provide the basic information about our database like Database name, Storage Space, Server name, etc.

Database Creation

Database Creation

Database name : The valid name of our SQL Database ( We have given the Database name as “AzureSQLDB” ).

Subscription : We can select our Azure subscription for SQL Database creation.

Resource group : We can create a new resource group or choose from an existing one ( We have selected our existing resource group as “WebAppAzureSql” ).

Select source : We select Blank database ( This will create a blank database ). The following Select source categories are available in the SQL Database.

  1. Blank database – This will create a blank database.
  2. AdventureWorksLT This will generate an AdventureWorksLT sample schema.
  3. Backup : Create a new database from an existing backup.

Server

Under the server, we need to configure required settings.

  1. Server name : Any globally unique name we can give which will store our database information.
  2. Server admin login : Create our server admin name for future access.
  3. Password & Confirm Password : Create our server admin password for future access.
  4. Location : Choose an available location that will be more suitable for our requirement.

Once we enter all the “Server” details then Click on “Select”.

Pricing tier : Select an appropriate pricing tier for our requirement.

Collation : Create a name for the Collation (Collation defines the rules that sort and compare the data and cannot be changed after the database creation ).

Click on “Create” to provision the database.

Database provisioned successfully!! Go to the Dashboard and Click on the Sql Database ( “AzureSqlDB” ).

DashBoard

DashBoard

Now open the details page of SQL database ( “Overview” ). Here, we can see the Server name and other created details. We can access the SQL Database through this Server name in Azure Portal & Local MSSQL.

Create a server-level firewall rule

We need to setup a firewall rule for accessing our Azure Sql database in Azure Portal. So the SQL Database service creates a firewall at the server-level that prevents external applications and tools from connecting to the server. If we not set up any firewall rule in Azure SQL database portal, then we will get the following error.

Setup Firewall Rule

Click on the “Set server Firewall” in the Overview Section.

The “Client IP address” automatically fetched into the Firewall Setting. We just copy the IP Address and Add the START IP & END IP Section (This will add based on more than one IP address ). Set a “RULE NAME” of our Firewall. Once it’s all done then the information saves automatically into the firewall rule section.

SQL Database Login in Azure Portal

Click on the “Query editor ( preview)” and Click on the “Login” and it will display a Login portal for accessing our Azure SQL Database in Azure. So give or appropriate database credential into it. Once it’s done Click on “OK” button.

Output 1

It will open our Azure Sql Database Query Editor window with Database details.

Output 2

Accessing through our Local “Microsoft SQL Server Management Studio”.

Reference

See Also

You can download other ASP.NET Core & Azure source codes from MSDN Code, using the link, mentioned below.

Summary

In this article, we are going to create and connect Azure SQL database to the Azure portal. I hope this article is useful for all Azure beginners & experts.

Channel Configuration : Azure Bot Service to Slack Application


Introduction

This article explains how to configure Azure Bot Service to Slack Applications. So, before reading this article, please read our previous article related to Create and Connect a chat bot with Azure Bot Service. Then, we will get a clear idea of how to create a Bot service in Azure.

Create a Web App Bot in Azure

Click on “New” on the left side menu and it will open an Azure Marketplace. There, we can see the list of services. Click “AI + Cognitive Services” then click on the “Web App Bot” for your bot service app.

Bot Service

Fill the following details and add the location based on your client location or your Geolocation.

Once the build is successful, click on the “Dashboard” and we can see that the “menothbotdemo” bot is created in the All resources list. Bot is ready for use!

Create a Slack Application for our bot

First, we need to create a workspace in Slack Account. Check the following link to create a Slack Account: New slack account

Create an app and assign a Development Slack team or Slack Workspace

  1. Click on the url https://api.slack.com/apps. Then, click on the “Create New App” !!.

Once the Slack workspace is created, then only we can create a slack application under the Workspace. Now, we are going to create and assign our slack app name into the Workspace. We have given our App a name as “menothbotdemo”.

Click on the “Create App” button. Then, Slack will create our app and generate a Client ID and Client Secret. We can use these IDs for channel configuration in Azure Web App bot.

Add a new Redirect URL

Click on the “OAuth & Permission” tab in the left panel. Then, add the redirect URLs as “https://slack.botframework.com&#8221; and save it properly.

Create Bot Users

Click on the “Bot Users” tab in the left panel. Then, click on “Add a Bot User”. In this section, we can give our bot “Display name”. For example, we created our bot user’s name as “menothbotdemo”. If we want our bot to always show as Online, then click on the “On” button. After that, click “Add Bot User” button.

Event Subscriptions

  1. Select “Event Subscriptions” tab in the left panel.
  2. Click Enable Events to On.
  3. In the “Request URL” we need to add the following URL to our “Bot Handle Name”.

https://slack.botframework.com/api/Events/{bot handle name}

The “Bot Handle” name we will get inside the “Web App Bot ( we created our web app as “menothbotdemo”)” Settings.

Finally, we can add the Request URL inside the Event Subscriptions.

4.  In Subscribe to Bot Events, click “Add Bot User Event”.

5. In the list of events, click “Add Bot User Event” and select the following event name.

Subscribe to Bot Events

6. Click “Save Changes”.

Configure Interactive Messages ( Optional )

  1. Select the “Interactive Components” tab and click “Enable Interactive Components”.
  2. Enter https://slack.botframework.com/api/Actions as the request URL.
  3. Click the “Enable Interactive Messages” button, and then click the “Save Changes” button.

App Credentials

Select the “Basic Information” tab and then we will get the ClientID & Client Secret & Verification Token for our channel configuration in Azure Bot Service.

Channel Configuration

There is a very simple way to connect our bot service app to Slack in Azure. Just follow the following steps.

Click on the “Channels” menu on the left side option. Then, it will open a window with channel details where you can see “More channels” options. Then, select “Slack” in the channels list.

Add the following Slack App ( Already Created Slack App ) credentials into the Azure Slack configuration section.

  • ClientID
  • Client Seceret
  • Verification Token

Once the configuration is done, we can see our Slack configured into the channel.

C# Code

We have done some changes in the default code in bot service.

using System;
using System.Threading.Tasks;

using Microsoft.Bot.Connector;
using Microsoft.Bot.Builder.Dialogs;
using System.Net.Http;

namespace Microsoft.Bot.Sample.SimpleEchoBot
{
[Serializable]
public class EchoDialog : IDialog<object>
{
protected int count = 1;

public async Task StartAsync(IDialogContext context)
{
context.Wait(MessageReceivedAsync);
}

public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> argument)
{
var message = await argument;

if (message.Text == "reset")
{
PromptDialog.Confirm(
context,
AfterResetAsync,
"Are you sure you want to reset the count?",
"Didn't get that!",
promptStyle: PromptStyle.Auto);
}
else if (message.Text == "Hi")
{
await context.PostAsync($"{this.count++}: Slack Configured in Bot App !!");
context.Wait(MessageReceivedAsync);
}
else
{
await context.PostAsync($"{this.count++}: You said {message.Text}");
context.Wait(MessageReceivedAsync);
}
}

public async Task AfterResetAsync(IDialogContext context, IAwaitable<bool> argument)
{
var confirm = await argument;
if (confirm)
{
this.count = 1;
await context.PostAsync("Reset count.");
}
else
{
await context.PostAsync("Did not reset count.");
}
context.Wait(MessageReceivedAsync);
}

}
}

Output

Reference

See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below.

Summary

We learned how to configure Azure Bot Service to Slack application. I hope this article is useful for all Azure beginners.

Create and Connect a chat bot with Azure Bot Service


Introduction

This article explains how to Create and Connect a chat bot with Azure Bot Service.

Azure Account

First, we need to create an account on the Azure portal. Only then can we host the application in the cloud environment. So, please check the following steps to create an Azure account.

Azure Account Registration

Create an account through this link to  click here.

Web App Bot

  1. Click on “New” on the left side menu and it will open an Azure Marketplace , there we can see list of services so click on “AI + Cognitive Services” then click on the “Web App Bot” for our bot service app.

BOT Service Registration

  1. Bot name  : The display name of our bot service and that appears in channels and directories. We can change this name at any time.
  2. Subscription : We can select our Azure subscription for chat bot service.
  3. Resource group : We can create a new resource group or choose from an existing one ( We selected our existing resource group as “AzureDemo” ).
  4. Location : We can select our location of resource group. The best thing is we can choose a location closest to our customer. The location cannot be changed once the bot is created.
  5. Pricing tier : Select a pricing tier of bot service.
  6. App name : The unique URL name of our bot service , We given “menothbot” as our App name and the URL is look like this : http://menothbot.azurewebsites.net/
  7. Bot template : There are two templates available in bot C# and Node.js. We can choose any of the template and that will create a echo bot.
  8. App service plan/Location : We can choose a best service plan that closest to our customer.
  9. Azure Storage : We can create a new data storage account or use an existing one. By default, the bot will use Table Storage.
  10. Application Insights : This will provide service-level and instrumentation data like traffic, latency, and integrations. We can switch on or off this option.

11.Click on the “Create” button and wait for the build success.

12. Once the build is succeeded, then click on the “Dashboard” and we can see “menothbot” bot is created in the All resources list . Bot is ready for use !!.

Online Code Editor

  1. Click on the “menothbot” bot in dashboard window , Then After we can see a list option available for our bot service. So click on the “Build” option in left side menu and it will open multiple option in right side. Just click on “Open online code editor” link.

2. Online code editor will open a source code window of our bot service app. So we can edit and add code in this section and currently it will display the default “echo bot” code of our bot service. Click on “WWWROOT -> Dialogs -> EchoDialog.cs”.

3. If you made any changes in the online code editor then click on the “build console” option on the left side menu and run it “build.cmd” command for the execution and deployment of the code.

Test in Web Chat

We can quickly test our bot through “Test in Web Chat” option. , So just click on the “Test in Web Chat” in the left side menu and it will open a chat bot on right side window. Here it will display few messages that we already added in the “EchoDialog.cs” in online code editor.

Code :

using System;
using System.Threading.Tasks;
using Microsoft.Bot.Connector;
using Microsoft.Bot.Builder.Dialogs;
using System.Net.Http;

namespace Microsoft.Bot.Sample.SimpleEchoBot
{
[Serializable]
public class EchoDialog : IDialog<object>
{
protected int count = 1;

public async Task StartAsync(IDialogContext context)
{
context.Wait(MessageReceivedAsync);
}

public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> argument)
{
var message = await argument;

if (message.Text == "reset")
{
PromptDialog.Confirm(
context,
AfterResetAsync,
"Are you sure you want to reset the count?",
"Didn't get that!",
promptStyle: PromptStyle.Auto);
}
else if (message.Text == "hi")
{
await context.PostAsync($"{this.count++}: Hi , How may I assist you ?");
context.Wait(MessageReceivedAsync);
}
else if (message.Text == "how are you ?")
{
await context.PostAsync($"{this.count++}: fine , What about u ?");
context.Wait(MessageReceivedAsync);
}
else if (message.Text == "hello")
{
await context.PostAsync($"{this.count++}: Hello , Tell Me !!");
context.Wait(MessageReceivedAsync);
}
else
{
await context.PostAsync($"{this.count++}: You said {message.Text} , This is Azure Bot Service !! Thank You All !!
by
RajeeshMenoth !! ");
context.Wait(MessageReceivedAsync);
}
}

public async Task AfterResetAsync(IDialogContext context, IAwaitable<bool> argument)
{
var confirm = await argument;
if (confirm)
{
this.count = 1;
await context.PostAsync("Reset count.");
}
else
{
await context.PostAsync("Did not reset count.");
}
context.Wait(MessageReceivedAsync);
}

}
}

 Connect a bot to Web Chat

This is very simple way to connect our bot service app to Web Chat in Azure. Please check the following steps !!.

  1. Click on the “Channels” menu in the left side option. Then it will open a window with channels details there you can see edit option in “Web Chat” channel.

2. Click on the edit option in “Web Chat” channel and It will display two “Secret Keys” with Iframe code. So choose the first “Secret Key” and add it on iframe code.

iframe code

Copy paste your iframe code in your html code and add the secret key available in the web chat edit option . Then it will display the Web chat bot in your app.

<iframe src='https://webchat.botframework.com/embed/menothbot?s=YOUR_SECRET_HERE'></iframe>

Output

Reference

See Also

You can download other ASP.NET Core source codes from MSDN Code, using the link, mentioned below.

Summary

We learned how to Create and Connect a chat bot with Azure Bot Service. I hope this article is useful for all Azure chat bot beginners.

 

%d bloggers like this: