Custom Validation
Write your own validation code, making it possible to perform complex checks, such as validating data against external databases or APIs.
Available after impler version 0.9.1
To implement custom validation logic, you can use the code editor provided in the Validator
tab of the Import Details page.
Your custom validation function should adhere to this structure:
The code
function must have the name code
, and it can return an array of error records if validation failures occur. Impler will call this code
function when performing custom validation.
Parameters provided to the `code` Function
The params
object provided to your custom validation function contains essential information about the import process and the data being validated. It follows this structure:
uploadId
is the id of import happening at the moment.extra
is thestring
,number
orjson
provided at the time of import.fileName
is the name of file that is being imported.data
is the array of records, where each record follows this strcuture:index
of the recordrecord
the actual record object being importederrors
is an key-value pair of errors for key ofrecord
if there is anyisValid
flag indicating whether current record has errors or not
totalRecords
number indicating totalRecords being importedchunkSize
number indicating current size of records
Returned Error Records
Error records returned from your custom validation code should follow this structure:
In the case of all records passing validation, return an empty array []
.
Example, Validating data against database data
Here's an example of how to validate data against data from a database using the Axios library:
Features
Access to variables inside code editor
Developers can access schema keys by pressing Ctrl + Space
to display a list of available variables and selecting from it.
Additionally, the suggestion box contains variables available in params
, such as params.uploadId
, params.fileName
, and more.
Performing HTTP calls
Your custom validation code can make HTTP calls using libraries like Axios. For example, you can retrieve data from your own API and validate it against the imported data.
Validation execution in chunks
The custom code function is executed in chunks, typically in batches of 500 records.
For example, if you are validating 10,000 records, the function will be called 20 times to validate the data in manageable segments.
Combining with static validation
Error records generated from your custom validation code are merged with errors generated from Impler's static validation processes, creating a comprehensive error report.
Scalable Architecture
Impler's custom validation functionality is built on a scalable architecture, allowing you to handle and validate large datasets, even in the millions.
With custom validation in Impler, you have the flexibility to implement intricate validation logic, ensuring data integrity and quality in your import processes.
This feature empowers you to customize your imports to meet your specific needs, making complex data imports more accessible and efficient.
Have any doubts? Shoot us a message directly on Discord.
Last updated