Azure CoGNITIVE SeaRCH Tutorial

Azure COGNITIVE SEARCH

In this tutorial we will start with a background on Azure search, moving forward will discuss its offerings and then will conclude by creating a full fledged azure search.

Azure search is a service provided by Azure that can ease the work needed to implement a search your website. It can ingest the data from multiple data sources like Azure SQL db, blob storage, data lake, etc. Further it provides us with options to add azure conginitive abilities on the incoming stream of data to capture insights.

Indexes & Indexers

Indexes in the context of azure search is nothing but your data search index. For each field that exist in an index you can configure it to be retrievable, filterable, searchable etc. Azure search uses Odata(check out Odata tutorial) as its backend, so every operation that exist there in the Odata can be applied in azure search index.

On the other hand, Indexer can be treated as the middle-ware acting between data source and index i.e. it performs the job of data syncing between data source and azure search index. Data updates can be managed through a high-watermark column, for example a timestamp that get updated to a higher value after every update to row. Similarly deletions can be captured by using a soft delete column. Further you can specify the interval at which indexer should run data syncing job with a minimum interval of 5 minutes.

Many a times a this question comes up - How to delete all documents present in an index?

To answer that, there is no easy way to do that as of now. Either you will have to delete the index itself or will have code a job to make use of DELETE calls to delete all documents present in the index.

Resource structure

Azure search has the concept of dedicated resources assigned as search units. The operation cost(Detailed Azure Search Pricing) associated with azure search is a linear function of the number of units assigned. Check out this link for an in-depth understanding. To give out some relative benchmarks numbers, a single unit can handle a throughput of 16/sec GET requests where as 8 units can do around 150/sec without any call fail. It is advisable to determine your requirements for making effective decision on number of units to use.

Creating an Azure Cognitive Search Service

To get started with creating azure search service, find search service in the list of available services

Configuring Search Service

After clicking on create, you will see window like in below image. In there, fill up all the details. Primarily it asks for you subscription, the resource group to which you want your service to be associated, a unique name (this will be used for the url of your search service, as in this case it will be https://azure-search-demo.search.windows.net ) and then the search service plan.

From here on either you can just click "Review + create" to skip all other configurations or can click Next button for Scale. We will go ahead with configuring scale as well. If you see the image below, this step allows us to configure number of replicas and partitions for the search service. Note that the these numbers defines your total number of units, to put it mathematically, No. of units = replicas x partitions.

On the next step you can configure the networking for the service and can select either to keep your service publicaly accessible or private. At the last step you can review your selected configurations and click create button.

This operation will take some time. After successful creation, you will be able to see the newly created service listed under search services.