Chapter 9. Adding Specialized Searches

The previous chapter explored the relationship between academic papers and journals, and we learned how to search for academic papers. This chapter moves on to the last of the top-level APIs, Search. In this chapter, we will learn how to search for web content. We will see how we can search for the latest news with certain keywords or categories. Further on, we will search for images and videos, and learn how to automatically suggest search queries for the end user. By the end of this chapter, we will be introduced to Bing Visual Search and find out how to create customized search experiences by using Bing Custom Search.

In this chapter, we will learn about the following topics:

Searching the web using the smart-house application

The Bing Web Search API provides us with a search experience similar to what we find at http://bing.com/search. It returns results that are relevant to any queries.

A response for any request to this API will contain web pages, images, videos, and news articles. In a typical scenario, this is the API you would use for any of these searches.

Note that, in a real-life scenario, all requests should be made from a server-side application, not from a client, as we do in this example.

Note

If you have not already done so, sign up for the Bing Web Search API at https://portal.azure.com. You can read more on the API at https://azure.microsoft.com/en-us/services/cognitive-services/bing-web-search-api/.

Before diving into the required technicalities for web searches, we are going to prepare our smart-house application.

Add a new View in the Views folder called BingSearchView.xaml. At the very least, this should contain two Combobox elements, one for the search type and one for the search filter. We need one TextBox element for our search query, as well as one Button element to execute the search. Finally, we need a TextBox element to display the search result.

To accompany the search types and search filter, we need to add a new file, called BingSearchTypes.cs, in the Model folder. Add the following two enums:

    public enum BingSearchType {
        Web, News, NewsCategory
    }

    public enum SafeSearch {
        Strict, Moderate, Off
    }

Adding this allows us to use both the Bing Web Search and Bing News Search APIs. The latter will be discussed later. The second enum, SafeSearch, will also be discussed in more detail later.

We need a new ViewModel. Add a new file called BingSearchViewModel.cs, to the ViewModels folder. In this, we need to add two string properties for our search query and the search results. We will also need one property of type BingSearchType to represent the selected search type. Also needed is a property of type SafeSearch to represent the selected safe-search filter. An ICommand property is needed for our button.

In addition, we need to be able to display the values from the previously created SafeSearch enums. This can be achieved by adding the following properties:

public IEnumerable<BingSearchType> AvailableSearchTypes  { 
    get { 
        return Enum.GetValues (typeof(BingSearchType)).Cast<BingSearchType>(); 
    } 
}

public IEnumerable<SafeSearch> SafeSearchFilter {
    get { 
        return Enum.GetValues(typeof(SafeSearch)).Cast<SafeSearch>(); 
    } 
}

We get all the values from each enum, and return them as an IEnumerable.

At the time of writing, none of the search APIs have any NuGet client packages, so we need to make the web requests ourselves. Copy the WebRequest.cs file we used in earlier chapters into the Model folder. Rename the file BingWebRequest.cs and the class BingWebRequest.

As all API calls are GET requests, we can simplify this class a bit. Remove the URL parameter from the constructor, and remove the _endpoint member completely. Doing so allows us to simplify the MakeRequest function, as follows:

public async Task<TResponse> MakeRequest<TResponse>(string url) {
    try {
        var request = new HttpRequestMessage(HttpMethod.Get, url);
                
        HttpResponseMessage response = await _httpClient.SendAsync(request);

        if (response.IsSuccessStatusCode) {
            string responseContent = null;

            if (response.Content != null)
                responseContent = await response.Content.ReadAsStringAsync();
                    
            if (!string.IsNullOrWhiteSpace(responseContent))
                return JsonConvert.DeserializeObject<TResponse> (responseContent, _settings);

            return default(TResponse);
        }

We do not need a request body, and have removed the TRequest and corresponding code. We have also hardcoded the HTTP method, and said that we will specify the complete URL endpoint when calling the function. The rest of the function should stay the same.

Note

Remember to add references to System.Web and System.Runtime.Serialization.

With that in place, we can move on. Make sure that the code compiles and executes before continuing.

To be able to use Bing Web Search, we need to create a new class. Add a new file called BingSearch.cs, to the Model folder.

We need to add a member of type BingWebRequest, which we will create in the constructor:

    private BingWebRequest _webRequest;

    public BingSearch() {
        _webRequest = new BingWebRequest("API_KEY_HERE");
    }

Create a new function called SearchWeb. This should accept two parameters, a string for the search query and a SafeSearch parameter. The function should be marked as async and return a Task<WebSearchResponse>. WebSearchResponse is a data contract we will learn more about presently:

public async Task<WebSearchResponse> SearchWeb(string query, SafeSearch safeSearch)
{
    string endpoint = string.Format("{0}{1}&safeSearch={2} &count=5&mkt=en-US","https://api.cognitive.microsoft.com/bing/v7.0/search?q=", query, safeSearch.ToString());

First, we construct our endpoint, which points us to the web search service. We make sure that we specify the query, q, the safeSearch selection, and the market, mkt. The latter two will be discussed presently in this chapter.

The only required parameter is the query string. This should not exceed a length of 1,500 characters. Other optional parameters are described in the following table:

Parameter

Description

responseFilter

A comma-delimited list of the result types to include in the response. If not specified, results will contain all types. Legal values include Computation, Images, News, RelatedSearches, SpellSuggestions, TimeZone, Videos, and WebPages.

setLang

A two-letter language code to specify the language for user interface strings.

textDecorations

Specifies whether or not the query term is highlighted in the results. Defaults to false.

textFormat

The type of formatting to apply to display strings. Can be either raw or HTML, with raw being the default.

There are a few more parameters apart from these ones. They are, however, common to all searches and will be discussed at the end of this chapter.

With the endpoint in place, we can move on:

    try {
        WebSearchResponse response = await _webRequest.MakeRequest<WebSearchResponse>(endpoint);

        return response;
    }
    catch (Exception ex) {
        Debug.WriteLine(ex.Message);
    }
return null;

With the newly constructed endpoint, we call MakeRequest on the _webRequest object. We specify the API key and endpoint as parameters to this call, and we expect a WebSearchResponse object as a response.

WebSearchResponse is a data contract, which we get by deserializing the JSON response from the API service. The top-level object will contain objects with the different result types. Look in the code samples provided in the file called BingSearchResponse.cs for a complete data contract.

Note

For a complete list of response objects from Bing Web Search, visit https://msdn.microsoft.com/en-us/library/dn760794.aspx#searchresponse.

Heading back to the BingSearchViewModel.cs file, we can add BingSearch as a member. The constructor should look as follows:

    public BingSearchViewModel() {
        _bingSearch = new BingSearch(); 
        SearchCommand = new DelegateCommand(Search, CanSearch);
    }

The CanSearch parameter should return true if we have any text entered into the search query text field. Search should, for now, look as follows:

    private async void Search(object obj) {
        switch (SelectedSearchType) { 
            case BingSearchType.Web:
                var webResponse = await _bingSearch.SearchWeb(SearchQuery, SelectedSafeSearchFilter);
                ParseWebSearchResponse(webResponse as WebSearchResponse);
                break;
            default:
                break;
        }
    }

We call the SearchWeb function on the _bingSearch object, passing on the SearchQuery and SelectedSafeSearchFilter properties as parameters. With a successful response, we send the response to a new function, ParseWebSearch:

private void ParseWebSearchResponse(WebSearchResponse webSearchResponse) {
    StringBuilder sb = new StringBuilder();

    Webpages webPages = webSearchResponse.webPages;

    foreach (WebValue website in webPages.value)
    {
        sb.AppendFormat("{0}\n", website.name);
        sb.AppendFormat("URL: {0}\n", website.displayUrl);
        sb.AppendFormat("About: {0}\n\n", website.snippet);
    }

    SearchResults = sb.ToString();
}

When we interpret the results from a web search, we are interested in the resulting webPages. For each web page, we want to output the name, the display URL, and a descriptive snippet.

A successful test run with the web search should present us with the following result:

Searching the web

Result objects from a web search contain a RankingResponse object. This will identify how the results will typically be displayed on a search website, ordered in a mainline and sidebar. In a production system, you should always aim to display results in the order specified by RankingResponse.

This can be done in two ways. One is to use the specified ID field to rank all of the results. The other way is a bit more complex. It involves splitting the results based on answer types and the result index.

Apart from the queries we have seen up to now, we can also query for computations (for instance, 2 + 2), time zone calculations, and related searches. These queries will result in JSON responses, which is a bit different from a regular web search.