Elasticsearch Ingest Node

Sometimes it is required to transform a document before indexing it. So, you need to use an ingest node to pre-process the document before actual indexing occurs. For example, if we want to rename a field and index it or remove a field from the document, all of these operations are handled by the Ingest node. In a cluster, all nodes have the ability to ingest, but it must be customized to be processed on by particular nodes. We need to follow a number of steps to ingest the nodes.

Step involved

Basically, two steps are involved in working of ingest node -

1. Creating a pipeline
2. Creating a doc

We will discuss both steps in detail.

Create a pipeline

First of all, we are required to create a pipeline that contains the processors. We will take an example for pipeline creation in which we will convert the content of enrolment field to integer type. To do this, run the following code in Elasticsearch plugin to execute the pipeline -

Copy Code

Response

If you get the same response as the below output, the code is executed successfully.

Screenshot

Create a Document

After pipeline creation, the next step is to create a document. Hence, we will create a document using the pipeline converter. Execute the following code -

Copy Code

Response

If you get the same response as the below output, code executed successfully.

Screenshot

Now, we will fetch the document from the index created above. For this, use GET command as shown below -

Copy Code

Response

If you get the same response as the below output, code is executed successfully.

Here, in this above output, you can see that the enrolment field has converted to Integer.

Without pipelining

Without pipeline creation, the integer type value will not convert into Integer. Look at the below example of index creation without using a pipeline.

Copy Code

Response

If you get the same response as the below output, the code is executed successfully.

Getting a document

Copy Code

Response

By running the above code, if you get the same response as the below output, code executed successfully.

Here, in this above output, you can see that the enrolment field value "25" has not been converted to Integer. It is a string printed between inverted comma " " without pipelining.