This article shows how a website search could be implemented using Angular 2, ASP.NET Core and Elasticsearch. Most users expect autocomplete and a flexible search like some of known search websites. When the user enters a char in the search input field, an autocomplete using a shingle token filter with a terms aggregation used to suggest possible search terms. When a term is selected, a match query request is sent and uses an edge ngram indexed field to search for hits or matches. Server side paging is then implemented to iterate though the results.
Code: https://github.com/damienbod/Angular2AutoCompleteAspNetCoreElasticsearch
ASP.NET Core server side search
The Elasticsearch index and queries was built using the ideas from these 2 excellent blogs, bilyachat and qbox.io. ElasticsearchCrud is used as the dotnet core client for Elasticsearch. To setup the index, a mapping needs to be defined as well as the index with the required settings analysis with filters, analyzers and tokenizers. See the Elasticsearch documentation for detailed information.
In this example, 2 custom analyzers are defined, one for the autocomplete and one for the search. The autocomplete analyzer uses a custom shingle token filter called autocompletefilter, a stopwords token filter, lowercase token filter and a stemmer token filter. The edge_ngram_search analyzer uses an edge ngram token filter and a lowercase filter.
private IndexDefinition CreateNewIndexDefinition()
{
return new IndexDefinition
{
IndexSettings =
{
Analysis = new Analysis
{
Filters =
{
CustomFilters = new List<AnalysisFilterBase>
{
new StemmerTokenFilter("stemmer"),
new ShingleTokenFilter("autocompletefilter")
{
MaxShingleSize = 5,
MinShingleSize = 2
},
new StopTokenFilter("stopwords"),
new EdgeNGramTokenFilter("edge_ngram_filter")
{
MaxGram = 20,
MinGram = 2
}
}
},
Analyzer =
{
Analyzers = new List<AnalyzerBase>
{
new CustomAnalyzer("edge_ngram_search")
{
Tokenizer = DefaultTokenizers.Standard,
Filter = new List<string> {DefaultTokenFilters.Lowercase, "edge_ngram_filter"},
CharFilter = new List<string> {DefaultCharFilters.HtmlStrip}
},
new CustomAnalyzer("autocomplete")
{
Tokenizer = DefaultTokenizers.Standard,
Filter = new List<string> {DefaultTokenFilters.Lowercase, "autocompletefilter", "stopwords", "stemmer"},
CharFilter = new List<string> {DefaultCharFilters.HtmlStrip}
},
new CustomAnalyzer("default")
{
Tokenizer = DefaultTokenizers.Standard,
Filter = new List<string> {DefaultTokenFilters.Lowercase, "stopwords", "stemmer"},
CharFilter = new List<string> {DefaultCharFilters.HtmlStrip}
}
}
}
}
},
};
}
The PersonCity is used to add and search for documents in Elasticsearch. The default index and type for this class using ElasticsearchCrud is personcitys and personcity.
public class PersonCity
{
public long Id { get; set; }
public string Name { get; set; }
public string FamilyName { get; set; }
public string Info { get; set; }
public string CityCountry { get; set; }
public string Metadata { get; set; }
public string Web { get; set; }
public string Github { get; set; }
public string Twitter { get; set; }
public string Mvp { get; set; }
}
A PersonCityMapping class is defined so that required mapping from the PersonCityMappingDto mapping class can be defined for the personcitys index and the personcity type. This class overrides the ElasticsearchMapping to define the index and type.
using System;
using ElasticsearchCRUD;
namespace SearchComponent
{
public class PersonCityMapping : ElasticsearchMapping
{
public override string GetIndexForType(Type type)
{
return "personcitys";
}
public override string GetDocumentType(Type type)
{
return "personcity";
}
}
}
The PersonCityMapping class is then used to map the C# type PersonCityMappingDto to the default index from the PersonCity class using the PersonCityMapping. The PersonCityMapping maps to the default index of the PersonCity class.
public PersonCitySearchProvider()
{
_elasticsearchMappingResolver.AddElasticSearchMappingForEntityType(typeof(PersonCityMappingDto), new PersonCityMapping());
_context = new ElasticsearchContext(ConnectionString, new ElasticsearchSerializerConfiguration(_elasticsearchMappingResolver))
{
TraceProvider = new ConsoleTraceProvider()
};
}
A specific mapping DTO class is used to define the mapping in Elasticsearch. This class is required, if a non default mapping is required in Elasticsearch. The class uses the ElasticsearchString attribute to define a copy mapping. The field in Elasticsearch should be copied to the autocomplete and the searchfield field when adding a new document. The searchfield and the autocomplete field uses the two analyzers which where defined in the index when adding data. This class is only used to define the type mapping in Elasticsearch.
using ElasticsearchCRUD.ContextAddDeleteUpdate.CoreTypeAttributes;
namespace SearchComponent
{
public class PersonCityMappingDto
{
public long Id { get; set; }
[ElasticsearchString(CopyToList = new[] { "autocomplete", "searchfield" })]
public string Name { get; set; }
[ElasticsearchString(CopyToList = new[] { "autocomplete", "searchfield" })]
public string FamilyName { get; set; }
[ElasticsearchString(CopyToList = new[] { "autocomplete", "searchfield" })]
public string Info { get; set; }
[ElasticsearchString(CopyToList = new[] { "autocomplete", "searchfield" })]
public string CityCountry { get; set; }
[ElasticsearchString(CopyToList = new[] { "autocomplete", "searchfield" })]
public string Metadata { get; set; }
public string Web { get; set; }
public string Github { get; set; }
public string Twitter { get; set; }
public string Mvp { get; set; }
[ElasticsearchString(Analyzer = "edge_ngram_search", SearchAnalyzer = "standard", TermVector = TermVector.yes)]
public string searchfield { get; set; }
[ElasticsearchString(Analyzer = "autocomplete")]
public string autocomplete { get; set; }
}
}
The IndexCreate method creates a new index and mapping in elasticsearch.
public void CreateIndex()
{
_context.IndexCreate<PersonCityMappingDto>(CreateNewIndexDefinition());
}
The Elasticsearch settings can be viewed using the HTTP GET:
http://localhost:9200/_settings
{
"personcitys": {
"settings": {
"index": {
"creation_date": "1477642409728",
"analysis": {
"filter": {
"stemmer": {
"type": "stemmer"
},
"autocompletefilter": {
"max_shingle_size": "5",
"min_shingle_size": "2",
"type": "shingle"
},
"stopwords": {
"type": "stop"
},
"edge_ngram_filter": {
"type": "edgeNGram",
"min_gram": "2",
"max_gram": "20"
}
},
"analyzer": {
"edge_ngram_search": {
"filter": ["lowercase",
"edge_ngram_filter"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "standard"
},
"autocomplete": {
"filter": ["lowercase",
"autocompletefilter",
"stopwords",
"stemmer"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "standard"
},
"default": {
"filter": ["lowercase",
"stopwords",
"stemmer"],
"char_filter": ["html_strip"],
"type": "custom",
"tokenizer": "standard"
}
}
},
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "TxS9hdy7SmGPr4FSSNaPiQ",
"version": {
"created": "2040199"
}
}
}
}
}
The Elasticsearch mapping can be viewed using the HTTP GET:
http://localhost:9200/_mapping
{
"personcitys": {
"mappings": {
"personcity": {
"properties": {
"autocomplete": {
"type": "string",
"analyzer": "autocomplete"
},
"citycountry": {
"type": "string",
"copy_to": ["autocomplete",
"searchfield"]
},
"familyname": {
"type": "string",
"copy_to": ["autocomplete",
"searchfield"]
},
"github": {
"type": "string"
},
"id": {
"type": "long"
},
"info": {
"type": "string",
"copy_to": ["autocomplete",
"searchfield"]
},
"metadata": {
"type": "string",
"copy_to": ["autocomplete",
"searchfield"]
},
"mvp": {
"type": "string"
},
"name": {
"type": "string",
"copy_to": ["autocomplete",
"searchfield"]
},
"searchfield": {
"type": "string",
"term_vector": "yes",
"analyzer": "edge_ngram_search",
"search_analyzer": "standard"
},
"twitter": {
"type": "string"
},
"web": {
"type": "string"
}
}
}
}
}
}
Now documents can be added using the PersonCity class which has no Elasticsearch definitions.
Autocomplete search
A terms aggregation search is used for the autocomplete request. The terms aggregation uses the autocomplete field which only exists in Elasticsearch. A list of strings is returned to the user from this request.
public IEnumerable<string> AutocompleteSearch(string term)
{
var search = new Search
{
Size = 0,
Aggs = new List<IAggs>
{
new TermsBucketAggregation("autocomplete", "autocomplete")
{
Order= new OrderAgg("_count", OrderEnum.desc),
Include = new IncludeExpression(term + ".*")
}
}
};
var items = _context.Search<PersonCity>(search);
var aggResult = items.PayloadResult.Aggregations.GetComplexValue<TermsBucketAggregationsResult>("autocomplete");
IEnumerable<string> results = aggResult.Buckets.Select(t => t.Key.ToString());
return results;
}
The request is sent to Elasticsearch as follows:
POST http://localhost:9200/personcitys/personcity/_search HTTP/1.1
Content-Type: application/json
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Content-Length: 124
Host: localhost:9200
{
"size": 0,
"aggs": {
"autocomplete": {
"terms": {
"field": "autocomplete",
"order": {
"_count": "desc"
},
"include": {
"pattern": "as.*"
}
}
}
}
}
Search Query
When an autocomplete string is selected, a search request is sent to Elasticsearch using a Match Query on the searchfield field which returns 10 hits from the 0 document. If the paging request is sent, the from value is a multiple of 10 depending on the page.
public PersonCitySearchResult Search(string term, int from)
{
var personCitySearchResult = new PersonCitySearchResult();
var search = new Search
{
Size = 10,
From = from,
Query = new Query(new MatchQuery("did_you_mean", term))
};
var results = _context.Search<PersonCity>(search);
personCitySearchResult.PersonCities = results.PayloadResult.Hits.HitsResult.Select(t => t.Source);
personCitySearchResult.Hits = results.PayloadResult.Hits.Total;
personCitySearchResult.Took = results.PayloadResult.Took;
return personCitySearchResult;
}
The search query as sent as follows:
POST http://localhost:9200/personcitys/personcity/_search HTTP/1.1
Content-Type: application/json
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Content-Length: 74
Host: localhost:9200
{
"from": 0,
"size": 10,
"query": {
"match": {
"searchfield": {
"query": "asp.net"
}
}
}
}
Angular 2 client side search
The Angular 2 client uses an autocomplete input control and then uses a ngFor to display all the search results. Bootstrap paging is used if more than 10 results are found for the search term.
<div class="panel-group">
<personcitysearch
*ngIf="IndexExists"
(onTermSelectedEvent)="onTermSelectedEvent($event)"
[disableAutocomplete]="!IndexExists">
</personcitysearch>
<em *ngIf="PersonCitySearchData.took > 0" style="font-size:smaller; color:lightgray;">
<span>Hits: {{PersonCitySearchData.hits}}</span>
</em><br />
<br />
<div *ngFor="let personCity of PersonCitySearchData.personCities">
<b><span>{{personCity.name}} {{personCity.familyName}} </span></b>
<a *ngIf="personCity.twitter" href="{{personCity.twitter}}">
<img src="assets/socialTwitter.png" />
</a>
<a *ngIf="personCity.github" href="{{personCity.github}}">
<img src="assets/github.png" />
</a>
<a *ngIf="personCity.mvp" href="{{personCity.mvp}}">
<img src="assets/mvp.png" width="24" />
</a><br />
<em style="font-size:large"><a href="{{personCity.web}}">{{personCity.web}}</a></em><br />
<em><span>{{personCity.metadata}}</span></em><br />
<span>{{personCity.info}}</span><br />
<br />
<br />
</div>
<ul class="pagination" *ngIf="ShowPaging">
<li><a (click)="PreviousPage()" >«</a></li>
<li><a *ngFor="let page of Pages" (click)="LoadDataForPage(page)">{{page}}</a></li>
<li><a (click)="NextPage()">»</a></li>
</ul>
</div>
The personcitysearch Angular 2 component implements the autocomplete functionality using the ng2-completer component. When a char is entered into the input, a HTTP request is sent to the server which in turns sends a request to the Elasticsearch server.
import { Component, Inject, EventEmitter, Input, Output, OnInit, AfterViewInit, ElementRef } from '@angular/core';
import { Http, Response } from "@angular/http";
import { Subscription } from 'rxjs/Subscription';
import { Observable } from 'rxjs/Observable';
import { Router } from '@angular/router';
import { Configuration } from '../app.constants';
import { PersoncityautocompleteDataService } from './personcityautocompleteService';
import { PersonCity } from '../model/personCity';
import { CompleterService, CompleterItem } from 'ng2-completer';
import './personcityautocomplete.component.scss';
@Component({
selector: 'personcityautocomplete',
template: `
<ng2-completer [dataService]="dataService" (selected)="onPersonCitySelected($event)" [minSearchLength]="0" [disableInput]="disableAutocomplete"></ng2-completer>
`
})
export class PersoncityautocompleteComponent implements OnInit {
constructor(private completerService: CompleterService, private http: Http, private _configuration: Configuration) {
this.dataService = new PersoncityautocompleteDataService(http, _configuration); ////completerService.local("name, info, familyName", 'name');
}
@Output() bindModelPersonCityChange = new EventEmitter<PersonCity>();
@Input() bindModelPersonCity: PersonCity;
@Input() disableAutocomplete: boolean = false;
private searchStr: string;
private dataService: PersoncityautocompleteDataService;
ngOnInit() {
console.log("ngOnInit PersoncityautocompleteComponent");
}
public onPersonCitySelected(selected: CompleterItem) {
console.log(selected);
this.bindModelPersonCityChange.emit(selected.originalObject);
}
}
And the data service for the CompleterService which is used by the ng2-completer component:
import { Http, Response } from "@angular/http";
import { Subject } from "rxjs/Subject";
import { CompleterData, CompleterItem } from 'ng2-completer';
import { Configuration } from '../app.constants';
export class PersoncityautocompleteDataService extends Subject<CompleterItem[]> implements CompleterData {
constructor(private http: Http, private _configuration: Configuration) {
super();
this.actionUrl = _configuration.Server + 'api/personcity/querystringsearch/';
}
private actionUrl: string;
public search(term: string): void {
this.http.get(this.actionUrl + term)
.map((res: Response) => {
// Convert the result to CompleterItem[]
let data = res.json();
let matches: CompleterItem[] = data.map((personcity: any) => {
return {
title: personcity.name,
description: personcity.familyName + ", " + personcity.cityCountry,
originalObject: personcity
}
});
this.next(matches);
})
.subscribe();
}
public cancel() {
// Handle cancel
}
}
The HomeSearchComponent implements the paging for the search results and and also displays the data. The SearchDataService implements the API calls to the MVC ASP.NET Core API service. The paging css uses bootstrap to display the data.
import { Observable } from 'rxjs/Observable';
import { Component, OnInit } from '@angular/core';
import { Http } from '@angular/http';
import { SearchDataService } from '../services/searchDataService';
import { PersonCity } from '../model/personCity';
import { PersonCitySearchResult } from '../model/personCitySearchResult';
import { PersoncitysearchComponent } from '../personcitysearch/personcitysearch.component';
@Component({
selector: 'homesearchcomponent',
templateUrl: 'homesearch.component.html',
providers: [SearchDataService]
})
export class HomeSearchComponent implements OnInit {
public message: string;
public PersonCitySearchData: PersonCitySearchResult;
public SelectedTerm: string;
public IndexExists: boolean = false;
constructor(private _dataService: SearchDataService, private _personcitysearchComponent: PersoncitysearchComponent) {
this.message = "Hello from HomeSearchComponent constructor";
this.SelectedTerm = "none";
this.PersonCitySearchData = new PersonCitySearchResult();
}
public onTermSelectedEvent(term: string) {
this.SelectedTerm = term;
this.findDataForSearchTerm(term, 0)
}
private findDataForSearchTerm(term: string, from: number) {
console.log("findDataForSearchTerm:" + term);
this._dataService.FindAllForTerm(term, from)
.subscribe((data) => {
console.log(data)
this.PersonCitySearchData = data;
this.configurePagingDisplay(this.PersonCitySearchData.hits);
},
error => console.log(error),
() => {
console.log('PersonCitySearch:findDataForSearchTerm completed');
}
);
}
ngOnInit() {
this._dataService
.IndexExists()
.subscribe(data => this.IndexExists = data,
error => console.log(error),
() => console.log('Get IndexExists complete'));
}
public ShowPaging: boolean = false;
public CurrentPage: number = 0;
public TotalHits: number = 0;
public PagesCount: number = 0;
public Pages: number[] = [];
public LoadDataForPage(page: number) {
var from = page * 10;
this.findDataForSearchTerm(this.SelectedTerm, from)
this.CurrentPage = page;
}
public NextPage() {
var page = this.CurrentPage;
console.log("TotalHits" + this.TotalHits + "NextPage: " + ((this.CurrentPage + 1) * 10) + "CurrentPage" + this.CurrentPage );
if (this.TotalHits > ((this.CurrentPage + 1) * 10)) {
page = this.CurrentPage + 1;
}
this.LoadDataForPage(page);
}
public PreviousPage(page: number) {
var page = this.CurrentPage;
if (this.CurrentPage > 0) {
page = this.CurrentPage - 1;
}
this.LoadDataForPage(page);
}
private configurePagingDisplay(hits: number) {
this.PagesCount = Math.floor(hits / 10);
this.Pages = [];
for (let i = 0; i <= this.PagesCount; i++) {
this.Pages.push((i));
}
this.TotalHits = hits;
if (this.PagesCount <= 1) {
this.ShowPaging = false;
} else {
this.ShowPaging = true;
}
}
}
Now when characters are entered into the search input, records are searched for and returned with the amount of hits for the term.
![searchaspnetcoreangular2_01]()
The paging can also be used, to do the server side paging.
![searchaspnetcoreangular2_02]()
The search functions like a web search with which we have come to expect. If different results, searches are required, the server side index creation, query types can be changed as needed. For example, the autocomplete suggestions could be replaced with a fuzzy search, or a query string search.
Links:
https://github.com/oferh/ng2-completer
https://github.com/damienbod/Angular2WebpackVisualStudio
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
https://www.elastic.co/products/elasticsearch
https://www.nuget.org/packages/ElasticsearchCRUD/
https://github.com/damienbod/ElasticsearchCRUD
http://www.bilyachat.com/2015/07/search-like-google-with-elasticsearch.html
http://stackoverflow.com/questions/29753971/elasticsearch-completion-suggest-search-with-multiple-word-inputs
http://rea.tech/implementing-autosuggest-in-elasticsearch/
https://qbox.io/blog/an-introduction-to-ngrams-in-elasticsearch
https://www.elastic.co/guide/en/elasticsearch/guide/current/_ngrams_for_partial_matching.html
![]()
![]()
