Converting Content to Attributes in Apache NiFi

In Apache NiFi, flowfiles are the fundamental data structures that carry data through the system. While flowfiles have content, they also have attribute (metadata that describes the content). Sometimes, it’s useful to convert specific pieces of content into attributes for easier processing and routing. This article will guide you through this process.

Step-by-step process :

1. Generate CSV Data

2. Convert CSV to JSON

3. Split Data

4. Extract Content (JSON) to Attributes

---

Set Up The Flow in NiFi

NiFi Flow Explanation:

1. Generate order details

   We begin by generating order details using the GenerateFlowFile processor, which creates flowfiles based on our sample data.

2. Convert CSV to JSON

   The next step is converting the data in the flowfile from CSV format to JSON format.

3. Split data

   We then split the JSON array into individual flowfiles, creating one flowfile for each entry.

4. Extract content (JSON) to attributes

   Finally, we extract the customer_id and order_id values into attributes for each flowfile.

---

Sample Data

Input Data: Order Details

customer_id,customer_name,customer_city,order_id,order_amount
C01,Billy,Jakarta,1011,250000
C01,Billy,Jakarta,1014,450000
C02,Depi,Lampung,1012,150000
C03,Indra,Pekanbaru,1013,300000

Convert Data: CSV to JSON

[ {
	"customer_id" : "C01",
	"customer_name" : "Billy",
	"customer_city" : "Jakarta",
	"order_id" : 1011,
	"order_amount" : 250000
}, {
	"customer_id" : "C01",
	"customer_name" : "Billy",
	"customer_city" : "Jakarta",
	"order_id" : 1014,
	"order_amount" : 450000
}, {
	"customer_id" : "C02",
	"customer_name" : "Depi",
	"customer_city" : "Lampung",
	"order_id" : 1012,
	"order_amount" : 150000
}, {
	"customer_id" : "C03",
	"customer_name" : "Indra",
	"customer_city" : "Pekanbaru",
	"order_id" : 1013,
	"order_amount" : 300000
} ]

Output Data: 4 Flowfiles

Content 1 :

Extracted Attributes :

Content 2 :

Extracted Attributes :

Content 3 :

Extracted Attributes :

Content 4 :

Extracted Attributes :

---

Processor Properties

• Generate order details

Processor Name: GenerateFlowFile

• Convert CSV to JSON

Processor Name: ConvertRecord

• Split Data

Processor Name: SplitJSON

• Extract Content (JSON) to Attributes

Processor Name: EvaluateJsonPath

---

Step-by-Step Video Guide

Converting Content to Attributes in Apache NiFi