Impler is built around configuring your import once and letting the user import spreadsheet data every time using Widget.

5 Easy Steps

1. Log in to your Account:

Head over to Impler by visiting https://web.impler.io and log in using your credentials

2. Create a New Import:

After logging in, you will land on the dashboard. Click on Create Import which will open the create modal asking you to give the appropriate name to the Import.

3. Add Columns to Your Import:

Once you've created a new import, you need to add columns to your import spreadsheet. The columns represent the data fields you wish to import.

The image represents a few fields and a button to add a new field.

4. Provide API Destination (Optional):

You can send Imported data to your backend application or Bubble.io application.

Enable Webhook destination and provide a REST api endpoint in case you want to send imported data to your application.

5. Import the data:

It's time to import your CSV or Excel file. We can do it in two ways:

1. Import from Dashboard: You can click on Import the button and the import widget will be opened. Which allows you to use data import functionality without any extra configuration.

2. Embed Widget into Application: You can embed CSV Excel Import widget into your application too, regardless of which tech stack you're using. Follow the steps mentioned in Snippet section of Importer, to embed a widget into your application. Read more in Importer

And that's it! With these five straightforward steps, you are well on your way to revolutionizing your data onboarding process and delivering an exceptional user experience.

Why?

The ability to import data is often needed in the application. It usually starts the same, reading .csv or .xlsx file and insert records into the database. But after a while, you'll find yourself looping over large files, validating rows, and providing support for file types that you've never heard of before.

Impler's goal is to help developers create an efficient and smooth data import experience between the product and its users. All with an easy-to-use API and outstanding developer experience.

Integrated into your Product

Integrating the import widget into your product can take just a few minutes. The feature-rich experience of the widget makes the data onboarding experience fun for users. Users can map columns and even review invalid records in their spreadsheets so that the right data gets imported.

As developers, we tend to care about the performance of the application, but following the growing number of file types support, mapping, and data validation become a burden on the team and make life hard.

Open Source

One of the reasons for making Impler open-source was to ensure privacy and give back to the community. We heard from many teams and we also felt How hard it's to give support for Data import functionality. So to make it accessible to everyone, we decided to build the Impler with awesome people like you.

How it works?

• Open the Importer and select the file you want to import.
• Select Header step will be opened. Which shows the first 15 rows for header selection. You can select the Header row of your choice or continue importing without headers by clicking on File does not have headers.
• Mapping will be based on the selected header row. Rows above the header row will not be imported.
• In the case of importing files without headers, the first row will be shown in the Mapping Step.

Once the user completes importing the spreadsheet in the widget, you access imported data immediately on the frontend application.

Enable Frontend Callback Destination

1. Open the Import and go to the Destination section.

2. Enable Get Imported Data in Frontend.

Get Imported Data on Frontend

Once the destination is enabled Impler will start sending Imported data on the frontend.

To access we can use onDataImported method at if using react or can check for aDATA_IMPORTED event as per if using HTML & JavaScript.

Impler provides the most generous way to integrate world-class CSV Excel Import functionality in any application. Impler only considers Imported Records as a metric to collect the amount and not additional parameters like no. of imports and no. of templates.

Working of Subscription in Impler

Impler assigns every user the Starter Plan, which includes 5,000 free records per month. Here’s how the subscription and billing process works:

• The Starter Plan Includes 5,000 free records per month. Importing more than 5,000 records incurs a charge relative to $1 for every 10,000 records.

• By default, the Starter Plan renews every month. If you have imported more than 5,000 records, you must pay the outstanding amount to renew your subscription. You can choose an appropriate plan to cover this.

• You must add a card before purchasing any plan. Adding a card creates a mandate of a certain amount, which helps the user to not provide payment details repeatedly and backend to deduct the amount at intervals.

• Plan upgrades take effect immediately. While Plan downgrades occur at the end of the current billing cycle.

• Users can cancel their plan anytime. Upon cancellation, the canceled plan does not renew at the end of the billing cycle, and the default Starter Plan is reactivated. Outstanding amounts are deducted at the end of the billing cycle after cancellation.

Users filling excel sheet often get's confused with "What value will go into this column?", "In which format I need to add phone number?", "What format to follow for date?", etc. We can answer these user questions with Column Descriptions.

How to provide column description?

Description input is available in Column Details Form, which get's opened by clicking on Validations Button.

Or by clicking on Edit Column button.

Which opens column modal which has input for column description.

How column description works?

Provided column description gets visible in review step while showing imported data to user.

It's possible to change border of tooltip by providing primaryColor while opening import widget.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

The Warning Column feature allows you to show helpful suggestions to users without blocking their data import. Unlike errors that stop the import process, warnings let users proceed while highlighting potential issues.

How it Works

Two Types of Validations

Key Benefits

• Key Benefit: Users are informed of any issues through warnings, but their uploads aren’t blocked.

Implementation Guide

Basic Structure

Every warning validation function follows this pattern:

const axios = require('axios');
exports.code = async (params) => {
  const result = [];
  // Loop through each item in the dataset
  for (const item of params.data) {
    result.push({
      index: item.index,
      warnings: {
        Country: 'Please specify your country',
      },
    });
  }

  return result;
};;

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

We heard issues like "I imported data but didn't receive callback", "Imported data is not showing in my application", "Validation code is not working as expected", etc. Also, "imported data not seen in the application", is something no one wants to face.

As a solution, Impler will send alert mail to users in the following scenarios,

1. Added webhook endpoint is not proper.

2. Application threw an error while sending imported data over the webhook.

3. Validation code throws an error during execution, which happens when data import is in progress.

Here is the sample mail,

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Purpose

Consider you have an ERP application that needs to import sales information. Our system provides a way to add extra information to sales information, like challan-no. Other users might have different columns.

In that case, fixing the import data structure is impossible. So we need to pass schema during run time when the user clicks on Import button to import the data.

You can find relevant implementation for react in and for HTML & HS in

Example Schema

[
  {
    name: 'Name',
    key: 'name',
    type: 'String'
  },
  {
    name: 'Phone',
    key: 'phone',
    type: 'Number',
    defaultValue: '#########'
  },
  {
    name: 'Country',
    key: 'country',
    type: 'String'
  }
]

Available properties for schema

Here are the keys

• name (required): The name of the column, will be displayed in the import widget during mapping.

• key (required): The key of the column, will be used to create an Excel file and match it with available headings in the file.

• type (optional): One of the types from String, Number, Date, Email, Regex, Select, and Any to override the existing type for the column.

• isRequired (optional): A boolean value indicating whether values in the current column are required.

• isUnique (optional): A boolean value indicating whether values in the current column must be unique.

• selectValues (optional): Select values indicating what are the possible values when the type is select.

• regex (optional): A regular expression to override when the type is regex.

• dateFormats (optional): Date formats indicating a list of formats acceptable for the date field. Required when type is Date. For example, ['dd/mm/yyyy','dd/mm/yy']

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Basic Example and Setup

const { showWidget, isImplerInitiated } = useImpler({
  projectId: "",
  templateId: "",
  accessToken: "",
  appearance: {
    primaryColor: '#FF6B35',
    fontFamily: 'Inter',
    borderRadius: '8px'
  }
});

AppearanceConfig type definition

export type AppearanceConfig = {
  widget?: {
    backgroundColor?: string;
  };
  primaryColor?: string;
  fontFamily?: string;
  borderRadius?: string;
  primaryButtonConfig?: ButtonConfig;
  secondaryButtonConfig?: ButtonConfig;
};

export type ButtonConfig = {
  backgroundColor?: string;
  buttonShadow?: string;
  textColor?: string;
  hoverBackground?: string;
  hoverBorderColor?: string;
  borderColor?: string;
  hoverTextColor?: string;
};

Note - All properties, including the appearance object, are optional. If none are provided, the widget will fall back to its default appearance.

ApperaranceConfig Property with Available Customization Options

Widget Properties:

• backgroundColor - Sets the main widget background color

• primaryColor - Main brand color used throughout the widget

• fontFamily - Font family for all text elements

• borderRadius - Corner rounding for all elements

Button Properties:

• backgroundColor - Button background color (default state)

• textColor - Button text color (default state)

• borderColor - Button border color (default state)

• buttonShadow - Drop shadow effect

• hoverBackground - Background color when hovering

• hoverTextColor - Text color when hovering

• hoverBorderColor - Border color when hovering

Quick Note

⚠️ Important: We're moving primaryColor inside the appearance object. The old way still works but will be deprecated in future releases.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Projects in Impler play a vital role in organizing imports effectively. You can invite your peers to the team and collaborate on Imports.

How Projects Work?

Open Projects Modal

Click on your project name on the top left corner of the Impler platform below the Impler logo to open Projects Modal.

Create Project

You can create unlimited projects in Impler. To create a new project write the project name in text input and click on Create. Upon creating a project you will be switched to a new project automatically.

Switch Project

Once the project is created it will be shown on the projects menu. As Owner only you can delete the project.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

List of available validators:

1. String Validator: This validator ensures that the column value is a string. String and Number are both valid values.

2. Number Validator: The Number validator verifies that the column value is a valid numeric entry. It only permits numerical values like 12 but will not allow 12.33 and john.

3. Double Validator: The Double validator verifies that the column value is either a Number or a Number with decimals. Valid values are 12 and 12.5 but john is not valid.

4. Email Validator: With the Email validator, you can ensure that the column value conforms to a valid email format. Valid values look like [email protected] while values like john and john.com are not valid.

5. Regex Validator: The Regex validator enables you to define a custom regular expression pattern that the column value must match.

6. Select Validator: Use the Select validator to restrict column values to a predefined set of options. For instance, you can use it to ensure that a "Gender" column contains only values like "Male" or "Female."

7. Any Validator: Any validator offers maximum flexibility by allowing any value in the column.

Validator Enhancements

• isRequired: Ensures that a value must exist in each cell for the column.

• isUnique: Ensures that the value is unique throughout the column.

• allowMultiSelect: Accepts multiple values separated by , for the cell.

It's possible to extend validation functionality to adjust according to your needs. Read more in Custom Validation.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Normal Flow

The normal importer flow has a higher number of steps to import the file and follows the below structure,

1. Download Sample File.

2. Write data into the file using spreadsheet editing software like Excel, Libre Office Calc, Numbers, etc.

3. Upload the file in which data is entered.

4. Select the header row.

5. Map columns.

6. Review data and fix invalid data if there is any.

7. Complete the Import.

This process takes a lot of time and involves many steps in the data import process. We introduced the facility to "Directly enter your data" to reduce the time and effort required to complete the import.

Directly enter your data

• Open the Importer and click on Directly enter your data.
• The spreadsheet editor will be opened which will show the column names as header and as per column types, columns will be shown. For example, for the date column, a date picker will be shown.
• Data will be validated instantly, so you can just continue editing the data and the importer will continue validating it in the background.

• While pasting the data directly from the spreadsheet, the validation will be performed after pasting is done.

• The user can not Finish Import if the data contains invalid values.

• On finish the data will be sent to frontend (Using Frontend Callback), Webhook (Using Webhook), or Bubble (Bubble.io Embed) as per the destination.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

i. Enable Multiselect from web portal

1. Click on the + button to add a new column.

2. Provide column Name and Type = Select.

1. Click on Validations.

1. We can provide allowed values in Select Values.

2. We can also enable Multi Select for select column. Which allows user to select multiple values in the cell.

1. Additionally we can sepcify , or ; as delimiter for multi-select values.

iii. Enabling Multiselect Dropdown using

We can enable the Multiselect feature by providing a custom schema with the allowMultiSelect flag in our frontend code.

Sample file generation and format

Impler creates .xlsx and .xlsm files based on the columns' configuration:

• .xlsx File: Generated if no columns have Multiselect enabled.

• .xlsm File: Generated if any column has Multiselect enabled.

Writing in .xlsm Files

When a column is marked as Multiselect, Impler appends #MULTI to the end of the column name in the Excel file. This allows users to select multiple options in the cell.

For detailed steps on how to write in .xlsm files refer to .

Data Format

Columns with the Multiselect feature will send an array of selected values when sent to application or .

Occasionally, Impler might add features that call for data or database schema alterations. This typically occurs when a feature requires certain data to be present on a database entity to function. To make these modifications to your database, you can utilize migrations.

Running Migrations

To run data migrations, enter the following sequence of commands in your terminal from the repository root:

Certain features can require more than one migration; in that case, you must do each migration in the following order.

Migrations History

Below you will find a list of migrations introduced in previous versions of Impler, alongside the migration path to use in the script above.

The Activity Page provides a comprehensive overview of your data imports, giving you insights into import statistics, trends, and detailed histories.

Let's explore the various sections and functionalities that the Activity Page has to offer.

1. Overview Statistics

The Activity Page greets you with key statistics that provide a quick snapshot of your data import activity:

• Total Imports: The count of all imports conducted using Impler.

• Total Records: The cumulative number of records imported across all your imports.

• Total Error Records: The count of records that encountered errors during the import process.

2. Import Count Trends (Bar Chart)

A graphical representation of the import count over the last seven days is displayed using a bar chart. This visual tool allows you to quickly identify trends and fluctuations in your import activity.

3. Import History Table

The Import History Table presents a paginated view of your import history, facilitating a deeper dive into each import's details. This table consists of the following columns:

• Import Id: A unique identifier for each import.

• Name: The name associated with the import.

• Date: The date on which the import occurred.

• Time: The time at which the import was initiated.

• Status: The status of the import, categorized as Upload, Mapped, Review, Confirmed, or Completed. The "Completed" status indicates that the imported data has been successfully sent to the application.

• Total Records: The total number of records in the import.

Filtering and Insight

The Import History Table offers filtering capabilities, allowing you to narrow down the displayed history by date and import name. This enables you to quickly locate specific imports and track their progress over time.

Import Insights

The Activity Page serves as a valuable resource for gaining insights into your import operations:

• Overview: Get a bird's-eye view of your import statistics.

• Trends: Identify patterns and trends in your import activity with the import count bar chart.

• Detailed History: Dive deep into each import's journey, from initiation to completion.

• Error Analysis: Monitor error records to address data quality issues.

Whether you're seeking a quick overview or a comprehensive analysis, the Activity Page equips you with the tools you need to assess your data import processes effectively.

Impler's Activity Page is designed to streamline your import tracking, optimize your workflow, and empower you with data-driven decision-making capabilities.

Impler is built upon scalable architecture to validate and process records of any size.

Separation of concern helps organizing code across various packages, libraries and apps. The idea is that app is divided into various parts, and each one is responsible for a spacific task.

Let's dive deep into building blocks of Impler.

The mental model

Widget

Widget is the main app where import happens. Widget provides UI to upload, map, validate and reivew data files. Widget gets embedded into iframe using embed script. It communicates with API to accomplish file processing work.

Queue Manager

Queue manager handles processing data. Once user completes file import, the command gets passed to Queue Manager app to start processing data and sending it to application in chunks.

Chunk contains chunked data along with import information. More information on

Authentication

API keys are used to authenticate APIs happening from import widget. Additionally developers can integrate Impler into their application by manually calling Impler API.

SDKs

SDKs removes the need of managing import widget manually. In case of missing SDK developer can take reference of to embed widget into application.

Three ways to freeze columns

We can freeze columns in many ways. Here is how,

i. Freeze the column while adding a new one

1. Click on the + button to add a new column.

2. Provide column Name, Key name and rest of the values based on your choice.

3. Check Frozen? checkbox for the column.

ii. Freezing already created column

1. Click on the Pencil (Edit) icon for any created column.

2. Check the Frozen? checkbox.

3. Click on Save.

iii. Freezing column with

We can freeze the column while providing a custom schema too with the isFrozen flag in our frontend code.

Effects on Sample File Generation

The generated sample file will have specified columns freeze on the left side. So they won't gets hidden if we scroll to the right.

Effects on Spreadsheet Editor

While importing data spreadsheet editor freezes two default Checkbox and Sr. No. Columns. Other mentioned columns get frozen as per schema.

How do I enable Image upload in Import?

The first step for image import is to add or update a column with the Image type. In case providing keep column type as Image and importer will be ready to import images.

How does Image Import work?

Once the import is configured to import data with Image. We can start importing data with Images. Here is the image import flow,

1. Generate a Template with Images,

The first step is to upload all images and generate a template with uploaded image names. All columns with the type Image will appear in Select Column dropdown and Uploaded images will be shown below Upload Image section.

Once all image files are selected, click on Generate Template to generate an Excel file holding the names of all uploaded images.

2. Upload Excel with data and image names.

Clicking on Generate Template will download an Excel file and Importer will move to the Upload section. You can enter data in the downloaded Excel file and select image names for image columns.

It's possible to generate a template again by clicking on Generate Template.

Once selected click on Map Columns.

3. Do mapping for columns

Click on Review Data to verify data getting imported.

4. Review Data

The review step checks if image columns contain the name of the uploaded image. If not shows a validation error.

Click on Re-Review Data to complete the import, once all records are valid.

5. Complete Import

6. Get Imported Data

Imported data will be sent to a webhook with an endpoint to download and access the uploaded file.

7. Download or Access the uploaded file

The download file endpoint is secured using accesskey which is available in Settings section on the left side. Here is the CURL for a review,

The .xlsm file format includes macro code that enables the Multiselect feature in Impler. Due to varying security protocols on different operating systems, there are specific steps to follow to ensure these macros run effectively.

Windows

Microsoft Excel on Windows restricts macro execution by default due to security concerns. Follow these steps to enable macros and use the Multiselect feature:

Steps to Enable Macros in Excel

1. Download the Sample File:

   1. First, download the sample file generated by Impler. If any column has Multiselect enabled, it will be in .xlsm format.

   2. By default, Microsoft Excel displays an Security Risk error for such files in its latest versions.
2. Mark the File as Safe:

   1. Navigate to the file location.

   2. Right-click on the .xlsm file and select Properties.

   3. In the Properties window, under the Security section, check the Unblock checkbox.

   4. Click Apply.
3. Enable Macros:

   • Open the file in Microsoft Excel.

   • An orange banner will appear at the top, indicating that macros are disabled.

   • Click on Enable Content it to allow the macros to run.

These steps will ensure that the macro code for the Multiselect feature is enabled, allowing you to select multiple options within the specified columns.

Linux

Linux users typically use LibreOffice Calc, which does not run macro code by default. However, you can still work with the Multiselect feature by manually entering options.

Steps for LibreOffice Calc

1. Open the .xlsm File:

   • Open the .xlsm file in LibreOffice Calc.

   • A warning message about macros will appear.
2. Manually Enter Options:

   • Since macros do not run in LibreOffice Calc, you need to manually type in the options for Multiselect.

   • For example, if you are entering countries, you can write New York,India,Canada in the cell.

By following these steps, you can effectively use the Multiselect feature in .xlsm files on both Windows and Linux.

Summary

Windows:

• Download the .xlsm file.

• Mark the file as safe through Properties.

• Enable macros in Excel to use the Multiselect feature.

Linux:

• Open the .xlsm file in LibreOffice Calc.

• Manually type the options in multi-select fields.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

How do I enable the automated import feature?

The process of automatic data import looks the same as manual data import, where the user uploads a file and impler imports the data. To convert any import to automated import we just need to switch its mode from Manual to Automatic. Once done your import is transformed from Manual to Automatic Import Mode.

How does Automated Import work?

When we switch import mode to Automatic, the Import widget gets transformed and instead of accepting the file from the user, the import widget asks the user for a source from where data will get imported on interval.

At the moment impler accepts RSS URL as a source from where data will be fetched at user-specified intervals. The embed process of automatic import is the same as manual import. Here is how automatic import works:

Step 1: Configure

The first step is to provide a source from where data will be imported at regular intervals. Here on RSS URL input, the user can provide the RSS URL and Impler will keep importing data from the URL.

Step 2: Map Columns

The second step is to Map appropriate columns from the RSS file. As RSS follows XML structure, Impler parses the RSS file, extracts all paths for possible values in the XML file, and returns headings. Users can pick an appropriate heading for the column.

Step 3: Schedule Timing

Impler imports data from specified sources at regular intervals. In the schedule step user defines the Interval. The ability to specify intervals in Minutes, Hours, Days, Months, and Days makes it limitless for a user to provide any timing.

Step 4: Confirm

At the end, impler confirms the automated import configuration. Which shows the source and when it will run.

Automated Imports are associated with users

Automated imports are associated with users. Users referred to here are users of end application who uses Impler. To mention userId we have to provide externalUserId extra parameters. As mentioned in Passing Extra Parameters (React) and (HTML & JS).

You can provide anything in externalUserId. ID or Email address that uniquely identifies the user is best suitable for value in externalUserId.

Showing Automated Imports in Application

Automated Imports refers to the user. It becomes obvious to show users import jobs into your application. Here are all the APIs to deal with import jobs in Impler.

Getting User Import Jobs

curl --location --request GET 'https://api.impler.io/v1/import-jobs/user/:externalUserId' \
--header 'accesskey: <ACCESS_TOKEN>'

Pause the User Import Job

curl --location --request GET 'https://api.impler.io/v1/import-job/user/:jobId/pause' \
--header 'accesskey: <ACCESS_TOKEN>'

Resume the User Import Job

curl --location --request GET 'https://api.impler.io/v1/import-job/user/:jobId/resume' \
--header 'accesskey: <ACCESS_TOKEN>'

Delete the User Import Job

curl --location --request GET 'https://api.impler.io/v1/import-job/user/:
externalUserId/:jobId' \
--header 'accesskey: <ACCESS_TOKEN>'

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

With inspiration from Birds of a feather flock together, you can invite your peers in Impler with different background, who are responsible for different work. Here are the details about which role has which capabilities in Impler.

Roles and Permissions

How Invitation Flow Works

1) Create a new account or Sign In

2) Click on `Invite` Button on Team Members

The person who created project will be on Admin role by default.

3) Provide Member Details and their Roles

Based on your current plan you can invite members to your team. Provide email IDs of your members and their roles. An invitation email will be sent to the invited members with a link to take action on the invitation.

4) Receive Invitation Link by Email or Copy from Sent Invitations

Invited members will get an email to accept the invitation or you can copy the invitation link from Sent Invitations section on Team Members.

5) Take Action on Invitation

On visiting the invitation link, if anyone is new to Impler, they can create an account or if they already have an account they can simply click on Accept Invitation.

Invitation Link View if a user is not logged In

While clicking on Signup

Email will be automatically filled while going to signup from invitation section.

Take action on the invitation

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Let's do a quick review of how to use @impler/react in your application.

Add Script

You copy this snippet to your code before the closing body tag.

It will add impler variable in window, so you can call its init and show method.

Install the Package

Add @impler/react in your application by running the following command.

Add Import Button

@impler/react provides headless useImpler hook that you can use to show an import widget in your application.

Customize Texts

You can customize any text in the import widget, here is the sample. A full list of available texts is available at .

Listening for Events

Usage Example

Applying App's Color Scheme

Your application may be developed by keeping dark and light modes in mind. So to let the Import widget show the widget overlay properly, you need to pass colorScheme in showWidget function.

Allowed values for colorScheme are light or dark.

in Sample File

You can provide default data to fill in the Sample file that gets generated from the Import widget. Default data act as a placeholder for the user to further add or update the data.

Here is an example of how to provide data to fill in the sample file,

Providing

The schema and output provided in the portal for any import behave as default values for any Import. It's possible to override the default schema and output, to adapt dynamic nature of Import.

Here is an example of how to provide dynamic schema and output,

Advanced Validations

You can provide in the column as validations key,

Using Typescript

If you're using typescript, you can leverage typescript types, like,

Passing Extra Parameters

It's an obvious need that we want to pass userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

The extra parameters get sent to an application during the webhook call.

Programmatically Closing Import Widget

Impler's import widget provides closeWidget a method that closes the import widget. Useful to close the import widget once data is imported.

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

Here is how,

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side. You can provide a string or async function that returns the token value.

Till the date, if imported spreadsheet contains missing value or any column is not mapped, the response uses {{key}} as placeholder. By utilizing Default Value functionality developer has more control over format of response they will receive.

Setting Default Values

Default value can be applied to columns either at the time of adding column or updating column. Wildcard values like null, undefined can also be used, which available in the dropdown.

Means developer can choose one of these predefined values or write default value as a string of their choice.

1. null: Represents the null value.

2. undefined: Represents an undefined value.

3. Empty String: Represents an empty string.

4. Empty Array ([]): Represents an empty array.

5. Boolean true: Represents the boolean value true.

6. Boolean false: Represents the boolean value false.

While providing custom schema you can mention with defaultValue key and put your desired string or appropriate value from mentioned table.

When default value gets applied?

The provided default value is used when imported data gets sent to application as response. The default value will be applied to column when,

1. Column Not Required and Not Selected During Mapping:

   • When a column is not marked as required, and the user chooses not to select the column during the mapping phase, the default value will be used in response.

2. Column Not Required and Contains Empty Value:

   • When a column is not marked as required, and the column contains an empty value, the default value will be applied to fill the gap.

Providing default value during

Web portal GUI makes it easy to assign default value to column. Based on needs you can provide default value while providing schema during runtime too.

Default Value example with

The Default Value functionality provides flexibility and control to developers when importing data. Whether dealing with optional columns or handling empty values, this feature enhances the overall data import experience.

Validations

Range

Validates that field's values for Number and Double columns fall within a min and max values. Either min, max or both must be defined. Values are inclusive.

Length

Validates that field's value for String column falls within min and max number of characters.

Either min, max or both must be defined. Values are inclusive.

Unique Across Multiple Fields

Validate uniqueness across multiple fields. To use it, place a unique_with validation on each field that you want to be part of the uniqueness validation, all sharing the same uniqueKey.

Here is an example set of fields using unique_with validations used to validate that the combination of department code and employee Id fields are unique.

Here is the example of unique_with validation in action,

Adding Validations

From Platform

1. Click on Validations while adding column or click on Edit to edit the column.
2. Assign validations as needed,

From Code

You can check the advanced validations sections in react( ), angular ) and javascript ()

Once the user completes importing the spreadsheet via the widget, Impler sends the imported data to the callback URL specified in the destination section.

If your callback URL is protected, you can enable header-based authentication. This ensures secure data transfer to your API endpoints.

How to add a Webhook Destination?

To configure a webhook destination in Impler:

1. Open the Import panel and navigate to the Destination section.

2. Enable the Webhook option.

3. Provide the REST API Endpoint of your application.

4. (Optional) Provide values for authentication and retry configuration.

🔒 Use authHeaderName and authHeaderValue if your endpoint requires authentication.

Chunk Size

You can configure the number of records sent per request by specifying a Chunk Size. This determines how many records Impler will include in a single webhook payload.

Authentication

To secure your API, Impler supports header-based authentication.

• Use the authHeaderName field to define the name of the header.

• Use the authHeaderValue prop in the import button to define its value.

These values are not visible to end users and are only transmitted when the webhook is triggered.

For integration help, refer to: Providing Authentication Header Value for React and for HTML & JS Embed.

Rate Limiting & Retry Mechanism

To ensure reliable data delivery and avoid data loss during transient errors or server-side rate limits, Impler supports a configurable retry mechanism. This is particularly useful when your webhook endpoints are subject to rate-limiting or temporary unavailability.

How Retry Works

When a webhook request to your endpoint fails (due to timeouts, rate-limiting responses like 429 Too Many Requests, or other non-2xx errors), Impler will retry the request based on the following two parameters:

• Retry Count: Defines how many times Impler will attempt to resend the data after the initial failure.

• Retry Interval: The interval (in milliseconds) between each retry attempt.

📝 Note: If not configured, Impler will not retry by default (RetryCount = 0).

Developer Notes

• The retries apply per chunk of data sent.

• Retries apply only for failed HTTP responses (non-2xx).

• If the final retry still fails, the chunk is considered undelivered, and a failure log is recorded.

• Ensure your API implements idempotency to handle retries safely without duplicate processing.

• If your server enforces rate limits (like 10 requests/second), set RetryInterval accordingly (e.g., 1000ms or more).

• Combine with Authentication Headers for secure, controlled access.

Webhook Data Format

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

To implement custom validation logic, you can use the code editor provided in the Validator tab of the Import Details page.

Your custom validation function should adhere to this structure:

The code function must have the name code, and it can return an array of error records if validation failures occur. Impler will call this code function when performing custom validation.

Parameters provided to the `code` Function

The params object provided to your custom validation function contains essential information about the import process and the data being validated. It follows this structure:

• uploadId is the id of import happening at the moment.

• extra is the string, number or json provided at the time of import.

• fileName is the name of file that is being imported.

• data is the array of records, where each record follows this strcuture:

  • index of the record

  • record the actual record object being imported

  • errors is an key-value pair of errors for key of record if there is any

  • isValid flag indicating whether current record has errors or not

• totalRecords number indicating totalRecords being imported

• chunkSize number indicating current size of records

Returned Error Records

Error records returned from your custom validation code should follow this structure:

In the case of all records passing validation, return an empty array [].

Example, Validating data against database data

Here's an example of how to validate data against data from a database using the Axios library:

Features

Access to variables inside code editor

Developers can access schema keys by pressing Ctrl + Space to display a list of available variables and selecting from it.

Additionally, the suggestion box contains variables available in params, such as params.uploadId, params.fileName, and more.

Performing HTTP calls

Your custom validation code can make HTTP calls using libraries like Axios. For example, you can retrieve data from your own API and validate it against the imported data.

Validation execution in chunks

The custom code function is executed in chunks, typically in batches of 500 records.

For example, if you are validating 10,000 records, the function will be called 20 times to validate the data in manageable segments.

Combining with static validation

Error records generated from your custom validation code are merged with errors generated from Impler's static validation processes, creating a comprehensive error report.

Scalable Architecture

Impler's custom validation functionality is built on a scalable architecture, allowing you to handle and validate large datasets, even in the millions.

With custom validation in Impler, you have the flexibility to implement intricate validation logic, ensuring data integrity and quality in your import processes.

This feature empowers you to customize your imports to meet your specific needs, making complex data imports more accessible and efficient.

Text customization is possible from the application using Customize Texts (React) and (HTML & JS). With this powerful feature, you can update every text on the Import widget. Useful for updating text on import widgets or even implementing localization.

Available Texts

Although you may not need to update every text on the import widget. Here is the complete text object which is used in Impler.

Texts contain variables, Like {total} which is used to put dynamic value. For example, 12K in the case of import confirmation.

{
  COMMON: {
    SORRY: 'Sorry!',
    CANCEL: 'Cancel',
    CLOSE_WIDGET: 'Close',
    UPLOAD_AGAIN: 'Upload Again',
  },
  STEPPER_TITLES: {
    GENERATE_TEMPLATE: 'Generate Template',
    UPLOAD_FILE: 'Upload',
    MAP_COLUMNS: 'Map Columns',
    REVIEW_DATA: 'Review',
    COMPLETE_IMPORT: 'Complete',
    CONFIGURE_JOB: 'Configure',
    SCHEDULE_JOB: 'Schedule',
    CONFIRM_JOB: 'Confirm',
  },
  FILE_DROP_AREA: {
    DROP_FILE: 'Drop and drop a file here or ',
    BROWSE_FILE: 'Browse from computer',
    IMAGE_FILE_SIZE:
      'Image size should be less than 5 MB. Supported formats are PNG, JPG and JPEG.',
    BRING_FILE: 'Bring any .csv or .xlsx file here to start Import',
    FILE_SELECTED: 'File selected successfully',
  },
  'PHASE0-1': {
    IMPORT_FILE: 'Import File',
    GENERATE_TEMPLATE: 'Generate Template',
    IMAGE_INFO_TITLE: 'Generate template with images',
    IMAGE_INFO_SUBTITLE:
      'Drag and drop images below for image columns and generate a template file containing names of uploaded images.',
  },
  PHASE1: {
    SELECT_TEMPLATE_NAME: 'Template',
    SELECT_TEMPLATE_NAME_PLACEHOLDER: 'Select Template',
    SELECT_TEMPLATE_REQUIRED_MSG: 'Please select a template from the list',

    SELECT_SHEET_NAME: 'Select sheet to Import',
    SELECT_SHEET_NAME_PLACEHOLDER: 'Select Excel sheet',
    SELECT_SHEET_CONFIRM: 'Select',
    SELECT_SHEET_REQUIRED_MSG: 'Please select sheet from the list',

    DOWNLOAD_SAMPLE: 'Download sample',
    GENERATE_TEMPLATE: 'Generate Template',
    SEE_MAPPING: 'See Mapping',

    SELECT_FILE_NAME: 'Select a file',
    SELECT_FILE_REQUIRED_MSG: 'Please select a file',
    SELECT_FILE_FORMAT_MSG:
      'File type not supported! Please select a .csv or .xlsx file.',

    TEMPLATE_NOT_FOUND_MSG:
      "We couldn't find the template you're importing! Please check the passed parameters.",
    INCOMPLETE_TEMPLATE_MSG:
      'This import does not have any columns. Please try again after some time!',
  },
  PHASE2: {
    REVIEW_DATA: 'Review Data',
    IN_SCHEMA_TITLE: 'Column in schema',
    IN_SHEET_TITLE: 'Column in your sheet',
    MAPPING_NOT_DONE_TEXT: 'Not Mapped',
    MAPPING_DONE_TEXT: 'Mapping Successful',
    MAPPING_FIELD_PLACEHOLDER: 'Select Field',
  },
  PHASE3: {
    COMPLETE: 'Complete',
    EXPORT_DATA: 'Export Data',
    RE_REVIEW_DATA: 'Re-Review Data',
    ALL_RECORDS_VALID_TITLE: ' All records are found valid!',
    ALL_RECORDS_VALID_DETAILS:
      'All {total} row(s) found valid! Would you like to complete the Import?',
    LABEL_ALL_RECORDS: `All {records}`,
    LABEL_VALID_RECORDS: `Valid {records}`,
    LABEL_INVALID_RECORDS: `Invalid {records}`,

    REPLACE: 'Replace',
    FIND_REPLACE: 'Find and Replace',
    ALL_COLUMNS_LABEL: 'All Columns',

    FIND_LABEL: 'Find',
    FIND_PLACEHOLDER: 'Empty Cell',

    REPLACE_LABEL: 'Replace',
    IN_COLUMN_LABEL: 'In Column',
    CASE_SENSITIVE_LABEL: 'Case Sensitive',
    MATCH_ENTIRE_LABEL: 'Match Entire Cell',
  },
  PHASE4: {
    TITLE: 'Bravo! {count} rows have been uploaded',
    SUB_TITLE:
      '{count} rows have been uploaded successfully and currently is in process, it will be ready shortly.',
    UPLOAD_AGAIN: 'Upload New File',
  },
  AUTOIMPORT_PHASE1: {
    MAPCOLUMN: 'Map Column',
  },
  AUTOIMPORT_PHASE2: {
    SCHEDULE: 'Schedule',
    IN_SCHEMA_TITLE: 'Column in schema',
    IN_FEED_TITLE: 'Key in your RSS feed ',
  },
  AUTOIMPORT_PHASE3: {
    CONFIRM: 'Confirm',
    INVALID_CRON_MESSAGE:
      'Expression values are incorrect. Please update values as per valid values below!',
  },
  DELETE_RECORDS_CONFIRMATION: {
    TITLE: `{total} rows will be deleted. Are you sure?`,
    DETAILS: 'This action cannot be undone.',
    CONFIRM_DELETE: 'Yes',
    CANCEL_DELETE: 'Cancel',
  },
  CLOSE_CONFIRMATION: {
    TITLE: `Are you sure? You will lose your work in progress.`,
    DETAILS: 'Your import is in progress, clicking <b>Yes</b> will reset it.',
    CONFIRM_CLOSE: 'Yes',
    CANCEL_CLOSE: 'No',
  },
}

Texts for Manual Import

When doing manual import of data the most commonly used texts are mentioned below. You can customize the same texts if you want to change language.

{
  "COMMON": {
    "SORRY": "Sorry!",
    "CANCEL": "Cancel",
    "CLOSE_WIDGET": "Close",
    "UPLOAD_AGAIN": "Upload Again"
  },
  "STEPPER_TITLES": {
    "GENERATE_TEMPLATE": "Generate Template",
    "UPLOAD_FILE": "Upload",
    "MAP_COLUMNS": "Map Columns",
    "REVIEW_DATA": "Review",
    "COMPLETE_IMPORT": "Complete",
    "CONFIGURE_JOB": "Configure",
    "SCHEDULE_JOB": "Schedule",
    "CONFIRM_JOB": "Confirm"
  },
  "FILE_DROP_AREA": {
    "DROP_FILE": "Drop and drop a file here or ",
    "BROWSE_FILE": "Browse from computer",
    "IMAGE_FILE_SIZE": "Image size should be less than 5 MB. Supported formats are PNG, JPG and JPEG.",
    "BRING_FILE": "Bring any .csv or .xlsx file here to start Import",
    "FILE_SELECTED": "File selected successfully"
  },
  "PHASE1": {
    "SELECT_TEMPLATE_NAME": "Template",
    "SELECT_TEMPLATE_NAME_PLACEHOLDER": "Select Template",
    "SELECT_TEMPLATE_REQUIRED_MSG": "Please select a template from the list",
    "SELECT_SHEET_NAME": "Select sheet to Import",
    "SELECT_SHEET_NAME_PLACEHOLDER": "Select Excel sheet",
    "SELECT_SHEET_CONFIRM": "Select",
    "SELECT_SHEET_REQUIRED_MSG": "Please select sheet from the list",
    "DOWNLOAD_SAMPLE": "Download sample",
    "GENERATE_TEMPLATE": "Generate Template",
    "SEE_MAPPING": "See Mapping",
    "SELECT_FILE_NAME": "Select a file",
    "SELECT_FILE_REQUIRED_MSG": "Please select a file",
    "SELECT_FILE_FORMAT_MSG": "File type not supported! Please select a .csv or .xlsx file.",
    "TEMPLATE_NOT_FOUND_MSG": "We couldn't find the template you're importing! Please check the passed parameters.",
    "INCOMPLETE_TEMPLATE_MSG": "This import does not have any columns. Please try again after some time!"
  },
  "PHASE2": {
    "REVIEW_DATA": "Review Data",
    "IN_SCHEMA_TITLE": "Column in schema",
    "IN_SHEET_TITLE": "Column in your sheet",
    "MAPPING_NOT_DONE_TEXT": "Not Mapped",
    "MAPPING_DONE_TEXT": "Mapping Successful",
    "MAPPING_FIELD_PLACEHOLDER": "Select Field"
  },
  "PHASE3": {
    "COMPLETE": "Complete",
    "EXPORT_DATA": "Export Data",
    "RE_REVIEW_DATA": "Re-Review Data",
    "ALL_RECORDS_VALID_TITLE": " All records are found valid!",
    "ALL_RECORDS_VALID_DETAILS": "All {total} row(s) found valid! Would you like to complete the Import?",
    "LABEL_ALL_RECORDS": "All {records}",
    "LABEL_VALID_RECORDS": "Valid {records}",
    "LABEL_INVALID_RECORDS": "Invalid {records}",
    "REPLACE": "Replace",
    "FIND_REPLACE": "Find and Replace",
    "ALL_COLUMNS_LABEL": "All Columns",
    "FIND_LABEL": "Find",
    "FIND_PLACEHOLDER": "Empty Cell",
    "REPLACE_LABEL": "Replace",
    "IN_COLUMN_LABEL": "In Column",
    "CASE_SENSITIVE_LABEL": "Case Sensitive",
    "MATCH_ENTIRE_LABEL": "Match Entire Cell"
  },
  "PHASE4": {
    "TITLE": "Bravo! {count} rows have been uploaded",
    "SUB_TITLE": "{count} rows have been uploaded successfully and currently is in process, it will be ready shortly.",
    "UPLOAD_AGAIN": "Upload New File"
  },
  "DELETE_RECORDS_CONFIRMATION": {
    "TITLE": "{total} rows will be deleted. Are you sure?",
    "DETAILS": "This action cannot be undone.",
    "CONFIRM_DELETE": "Yes",
    "CANCEL_DELETE": "Cancel"
  },
  "CLOSE_CONFIRMATION": {
    "TITLE": "Are you sure? You will lose your work in progress.",
    "DETAILS": "Your import is in progress, clicking <b>Yes</b> will reset it.",
    "CONFIRM_CLOSE": "Yes",
    "CANCEL_CLOSE": "No"
  }
}

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Before You Start: Prerequisites

i. Prepare Your Bubble App

First, we need to set up the data structure in Bubble and enable the API for Impler to connect.

1. Go to Data Tab: In your Bubble editor, navigate to the "Data" tab.

2. Select Data Types: Ensure you have the "Data types" sub-tab selected.

3. Define/Verify Your Data Type:

   • Identify the Data Type you want to import data into (e.g., Employees, Products, Contacts).

   • Make sure this Data Type has all the necessary fields you want to import (e.g., Full Name (text), Email (text), Date of Birth (date), Social Security Number (text)). Create any missing fields using the "Create a new field" button.

   • Crucially: This Data Type must be set to Publicly visible. Impler needs this permission to see the structure. Check the privacy rules for this data type if needed, but the type definition itself needs to be discoverable.

ii. Enable Bubble Data API & Get Key

1. Go to Settings Tab: Navigate to the Settings tab in your Bubble editor.

2. Go to API Sub-Tab: Click on the API sub-tab.

3. Enable Data API: Scroll down to the Public API endpoints section. Check the box labeled Enable Data API.

4. Expose Your Data Type: In the list below Enable Data API, find the Data Type you prepared in Step 1.1 (e.g., Employees) and check the box next to it. This allows API access specifically for this data type.

5. Generate API Private Key: Scroll further down to "API Tokens".

   • Enter a descriptive label for your key (e.g., "Impler Key").

   • Click "Generate a new API token".

   • Bubble will generate a Private key. Copy this key immediately and save it somewhere secure. You will need it shortly, and Bubble won't show it to you again.

2. Set Up Your Import Project

Now, let's configure Impler to receive data and send it to your Bubble app.

2.1) Create an Import

1. Log in to Impler: Go to and log in.

2. Create New Import: Click the Create Import button.

3. Name Your Import: Give it a clear name (e.g., "Bubble Employees Import") and click Create & Continue

2.2) Configure Bubble.io as the Destination

1. Go to Destination Tab: Inside your newly created import, click on the Destination tab.

2. Enable Bubble.io: Find the "Bubble.io" option and toggle it ON.

3. Fill in Bubble App Details:

   • Bubble App Name: Enter the name of your Bubble app. This is the part before .bubbleapps.io in your app's URL (e.g., if your app is mycoolapp.bubbleapps.io, enter mycoolapp).

   • Environment: Choose production (for your live app) or development (for your test version).

   • Custom Domain Name (Optional): If your app uses a custom domain (e.g., app.mycompany.com), enter it here. Otherwise, leave it blank.

   • API Private Key: Paste the Private key you generated and saved from Bubble (Step 1.2).

   • Datatype Name: Enter the exact name of the Bubble Data Type you prepared (e.g., Employees). Case-sensitivity might matter, so match it precisely.

4. Test and Save: Click "Test and Save". Impler will attempt to connect to your Bubble app using the details provided. Fix any errors if they occur.

2.3) Map Columns (Automatic)

1. Click "Map Columns": After saving the destination, click the "Map Columns" button (often located near the Bubble.io destination settings).

2. Auto-Mapping: Impler will attempt to automatically detect the fields from your Bubble Data Type and map them. Review the mappings to ensure they look correct. You can typically adjust these mappings later in the main Impler schema editor if needed.

3. Install the Impler Plugin

Let's add the importer interface to your Bubble app.

3.1) Install the Impler Plugin

1. Go to Plugins Tab: In your Bubble editor, go to the "Plugins" tab.

2. Add Plugins: Click the "+ Add plugins" button.

3. Search: Search for "CSV Excel Import Plugin Impler".

4. Install: Find the correct plugin and click "Install", then "Done".

3.2) Add the Impler Element to Your Page

1. Go to Design Tab: Navigate to the page where you want the import button/functionality.

2. Find Impler Element: In the "Visual Elements" section on the left panel, find "CSVExcelImporter" (the name might vary slightly based on the plugin version).

3. Drag onto Page: Drag and drop this element onto your page. Note: This element is usually invisible to the end-user; it just provides the necessary functions. You can place it anywhere, even outside the visible page area if you prefer.

3.3) Get Your Impler Project Credentials

1. Go Back to Impler: Return to your Impler project () and open the importer.

2. Click Integrate: Find and click the Integrate button (in the top-right of your Importer).

3. Copy Credentials: A modal or section will appear showing:

   • projectId

   • templateId

   • accessToken Copy these three values.

3.4) Configure the Impler Element in Bubble

1. Select Impler Element: Back in your Bubble editor (Design tab), double-click the CSVExcelImporter element you added in Step 3.2 to open its properties editor.

2. Paste Credentials: Paste the projectId, templateId, and accessToken you copied from Impler into the corresponding fields in the element's properties panel.

3.5) Initialize the Importer (Workflow)

We need to tell the Impler element to get ready when the page loads.

1. Go to the Workflow Tab: In your Bubble editor, switch to the "Workflow" tab.

2. Create Page Load Event: Click "Click here to add an event..." and choose "General" -> "Page is loaded".

3. Add Initialize Action: Click "Click here to add an action...", choose "Element Actions" -> "Initialize Importer" (the action name might vary slightly, select the one related to your CSVExcelImporter element).

4. Select Element: In the action properties, make sure the correct CSVExcelImporter element is selected.

3.6) Trigger the Importer (Workflow)

Users need a way to open the importer, typically via a button.

1. Add a Button: Go back to the "Design" tab and add a standard Bubble Button element to your page. Label it something like "Import Data" or "Upload CSV".

2. Add Button Workflow: Select the button and click "Start/Edit workflow".

3. Add Open Widget Action: Click "Click here to add an action...", choose "Element Actions" -> "Open Import Widget" (again, select the action associated with your CSVExcelImporter element).

4. Select Element: Ensure the correct CSVExcelImporter element is selected in the action properties.

4. Optional Advanced Configuration

4.1: Associating Imported Data with a User (UserId)

If you want to link the imported data to the specific Bubble user who uploaded it (e.g., storing the user on each imported Employee record):

1. In Bubble (Plugin Element):

   • Select the CSVExcelImporter element in the Design tab.

   • In its properties, find the User Id field.

   • Use Bubble's dynamic data capabilities to insert the relevant user ID. This is often Current User's unique id.
2. In Impler (Destination Output):

   • Go to your Impler project settings -> Destination tab.

   • Find the output schema or transformation settings (this might look like a code editor).

   • You need to tell Impler to include the User ID it receives from the plugin. Add a field mapping like this: "user": "{extra.userId}" (or potentially "Creator": "{extra.userId}" if your Bubble field is named Creator and is of type User). The exact syntax might depend on Impler's current interface, but the principle is to map the incoming extra.userId to the correct Bubble User field. Important: The field name on the left (user or Creator) MUST match the field name in your Bubble Data Type that holds the user information.

4.2) Customizing Importer Appearance (Theming)

1. In Bubble (Plugin Element):

   • Select the CSVExcelImporter element in the Design tab.

   • In its properties, find the Primary Color field.

   • Set a hex color code (e.g., #FF0000 for red) to customize the theme of the Impler widget to match your app's branding.

That's it! You've now integrated the Impler data importer into your Bubble application. Your users can now benefit from a streamlined way to upload data via CSV or Excel files. Remember to test thoroughly with different file types and data variations.

Here is the step-by-step guide to embed CSV Excel Importer into your angular application using @impler/angular.

Add Script

You copy this snippet to your code before the closing body tag.

<script type="text/javascript" src="https://embed.impler.io/embed.umd.min.js" async></script>

It will add impler variable in window, so you can call its init and show method.

Install the Package

Add @impler/angular in your application by running the following command.

npm i @impler/angular
# or
yarn add @impler/angular

Use Impler Service

@impler/angular provides ImplerService that you can use to initialize and show an importer to your application.

import { RouterOutlet } from '@angular/router';
import { isPlatformBrowser } from '@angular/common';
import { Component, Inject, PLATFORM_ID } from '@angular/core';
import { EventCalls, EventTypesEnum, ImplerService } from '@impler/angular';

@Component({
  selector: 'app-root',
  standalone: true,
  imports: [RouterOutlet],
  templateUrl: './app.component.html',
  styleUrl: './app.component.css',
})
export class AppComponent {
  title = 'impler-app';

  constructor(
    private implerService: ImplerService,
    @Inject(PLATFORM_ID) private platformId: Object
  ) {
    if (isPlatformBrowser(platformId)) {
      this.implerService.initializeImpler();
    }
  }
  public show(): void {
    this.implerService.showWidget({
      colorScheme: 'dark',
      projectId: '...',
      templateId: '...',
      accessToken: '...',
    });
  }
}

Customize Texts

You can customize any text in the import widget, here is the sample. All customizable texts are available at Available Texts.

public show(): void {
  this.implerService.showWidget({
    ...
    texts: {
      STEPPER_TITLES: {
        REVIEW_DATA: 'Check Data', // New Title
      },
    }
  });
}

Listening for Events

Usage Example

...
import { EventCalls, EventTypesEnum, ImplerService } from '@impler/angular';

@Component({
  ...
})
export class AppComponent {
  title = 'impler-app';

  constructor(
    private implerService: ImplerService,
    @Inject(PLATFORM_ID) private platformId: Object
  ) {
    if (isPlatformBrowser(platformId)) {
      this.implerService.initializeImpler();
      this.implerService.subscribeToWidgetEvents((eventData: EventCalls) => {
        switch (eventData.type) {
          case EventTypesEnum.DATA_IMPORTED:
            console.log('Data Imported', eventData.value);
            break;
          default:
            console.log(eventData);
            break;
        }
      });
    }
  }
}

Applying App's Color Scheme

Your application may be developed by keeping dark and light modes in mind. So to let the Import widget show the widget overlay properly, you need to pass colorScheme in showWidget function.

Allowed values for colorScheme are light or dark.

public show(): void {
  this.implerService.showWidget({
    ...
    colorScheme: 'dark',
  });
}

Data Seeding in Sample File

You can provide default data to fill in the Sample file that gets generated from the Import widget. Default data acts as a placeholder for the user to add or update the data further.

Here is an example of how to provide data to fill in the sample file,

public show(): void {
    this.implerService.showWidget({
      data: [
          { country: 'Canada' },
          { country: 'Australia' },
          { country: 'Germany' },
      ]
    });
}

Providing Runtime Schema

The schema and output provided in the web.impler.io portal for any import behave as default values for any Import. It's possible to override the default schema and output, to adapt a dynamic nature of Import.

Here is an example of how to provide dynamic schema and output,

public show(): void {
  this.implerService.showWidget({
      schema: [
          {
            key: 'country',
            name: 'Country',
            type: 'String'
          }
      ],
      output: {
        "%data%": {
          "country_id": "{{country}}"
        },
        "page": "{{page}}",
        "chunkSize": "{{chunkSize}}",
        "isInvalidRecords": "{{isInvalidRecords}}",
        "template": "{{template}}",
        "uploadId": "{{uploadId}}",
        "fileName": "{{fileName}}",
        "extra": "{{extra}}"
      }
  });
}

Advanced Validations

You can provide Advanced Validations in the column as validations key,

public show(): void {
  this.implerService.showWidget({
    schema: [
      {
        "key": "Department Code",
        "name": "Department Code",
        "type": "String",
        "validations": [
          {
            "validate": "unique_with",
            "uniqueKey": "Employee No"
          }
        ]
      },
      {
        "key": "Employee Id",
        "name": "Employee Id",
        "type": "Number",
        "validations": [
          {
            "validate": "unique_with",
            "uniqueKey": "Employee No"
          }
        ]
      }
    ]
  });
}

Using Typescript

If you're using typescript, you can leverage typescript types, like,

import { useImpler, ColumnTypes, ValidationTypes } from '@impler/angular';

public show(): void {
  this.implerService.showWidget({
    schema: [
        {
          key: 'country',
          name: 'Country',
          type: ColumnTypes.STRING,
          "validations": [
            {
              "validate": ValidationTypes.LENGTH,
              "min": 5,
              "max": 100,
              "errorMessage": "Country Name must be between 5 to 100 characters"
            }
          ]
        }
    ]
  });
}

Passing Extra Parameters

It's an obvious need that we want to pass parameters like userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

public show(): void {
  this.implerService.showWidget({
    extra: {
      userId: '4ddhodw3',
      time: new Date().this string()
    }
  });
}

The extra parameters get sent to an application during the webhook call.

Programmatically Closing Import Widget

Impler's import widget provides closeWidget a method that closes the import widget. Useful to close the import widget once data is imported.

public close(): void {
    this.implerService.closeWidget();
}

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

public show(): void {
    this.implerService.showWidget({
        title: "Employee Import"
    });
}

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

public show(): void {
    this.implerService.showWidget({
        primaryColor: '#5f45ff'
    });
}

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side. You can provide a string or async function that returns the token value.

public show(): void {
    this.implerService.showWidget({
        authHeaderValue: async () => {
            return "..."
        }
    });
}

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

There are chances that the system which is receiving data is not made or managed by us. But as a plugin or integration, we need to send imported data to that system. In that case, Impler allows changing the output format which follows the system's syntax.

Output Format Sample

{
  "%data%": {
    "month": "{{month}}",
    "day": "{{day}}",
    "AverageTemperatureFahr": "{{AverageTemperatureFahr}}",
    "AverageTemperatureUncertaintyFahr": "{{AverageTemperatureUncertaintyFahr}}",
    "city": "{{city}}",
    "country_id": "{{country_id}}",
    "Country": "{{Country}}",
    "Latitude": "{{Latitude}}",
    "Longitude": "{{Longitude}}"
  },
  "page": "{{page}}",
  "chunkSize": "{{chunkSize}}",
  "isInvalidRecords": "{{isInvalidRecords}}",
  "template": "{{template}}",
  "uploadId": "{{uploadId}}",
  "fileName": "{{fileName}}",
  "extra": "{{extra}}"
}

Usage Examples

Joining First Name and Last Name

{
  "%data%": {
    "FullName": "{{firstName}} {{lastName}}"
  },
  "page": "{{page}}",
  "chunkSize": "{{chunkSize}}",
  "isInvalidRecords": "{{isInvalidRecords}}",
  "template": "{{template}}",
  "uploadId": "{{uploadId}}",
  "fileName": "{{fileName}}",
  "extra": "{{extra}}"
}

Providing an Object for each value

{
  "%data%": {
    "city": {
      "label": "City",
      "id": "city",
      "value": "{{city}}"
    }
  }
}

Passing an Array for value

{
  "%data%": {
    "address": ["{{addressLine1}}","{{addressLine2}}", "{{state}}", "{{city}}", "{{pincode}}"]
  }
}

Output Format works differently for dynamic schema

If you're providing schema runtime from the application, in that case, default provided output isn't taken into consideration. Because columns defined in the output might not match with columns getting imported.

In that case, we need to provide schema runtime. We have listed examples for react in Providing Runtime Schema and for HTML & JS versions in Providing Runtime Schema.

This output customization capability empowers you to direct imported data to any destination and in any preferred format.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

Add Script

You copy this snippet to your code before the closing body tag.

It will add impler variable in window, so we can call its init and show methods later.

Add Import Button

Import widgets get opened when the user clicks on Import button. So let's add an import button,

Initialize Widget

Before the widget gets shown you have to call its init method. So write this code, after added embed script.

Here is the description of what we just did,

1. generateUuid the function generates UUID which helps add multiple import widgets on the same page.

2. init method on impler initializes impler widget.

3. readyCheckInterval calls impler isReady() method in the interval of 1 second to check whether the impler widget is initiated or not.

4. Once Impler is ready we remove disabled attribute from the import button, which the user can click to open the import widget.

Show Widget

After the initialization, when the user clicks on the Import button we will call show method to open the widget,

You can pass more parameters to show a method like schema, data, etc. Have a look at section on React embed.

That's it, your app is now ready to onboard user data into your application.

Customize Texts

You can customize any text in the import widget, here is the sample. A full list of available texts is available at .

Listening to Events

Impler instance provides message an event listener which gets called for various events happening with the widget. Here is how to attach event listener,

eventData follows the { type, value } structure.

Complete Code Example

in Sample File

You can provide default data to fill in the Sample Excel file that gets generated from the Import widget. Default data act as a placeholder for the user to further add or update the data in the file.

Here is an example of how to provide data to fill in the sample file,

Advanced Validations

You can provide in the column as validations key,

Providing

The schema and output provided in the portal for any import behave as default values for any Import. It's possible to override the default schema and output, to adapt dynamic nature of Import.

Here is an example of how to provide dynamic schema and output,

Passing Extra Parameters

It's an obvious need that we want to pass userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

The extra parameters get sent to an application during the webhook call.

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

Here is how,

Programmatically Closing the Import Widget

You can close the import widget programmatically by calling close method on impler instance,

Here is how,

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side.

Requirements

• Node.js version v18.13.0

• MongoDB

• RabbitMQ

• localstack (required for storing files)

• (Optional) pnpm - Needed if you want to install new packages

You can also run docker containers for MongoDB, RabbitMQ, and Localstatck.

Setup the project

After installing the required services on your machine, you can clone and set your forked version of the project:

• Fork Impler's repository. Clone or download your fork to your local machine.

• Run the initial setup command npm run setup:project to install and build all dependencies.

• Run the project locally using npm run start:dev

The npm run start:dev will start all of the services in parallel including APIs and Widget clients. if you only want to run parts of the platform, you can use the following run commands from the root project.

• start:api - Starts the API in watch mode

• start:dal - Builds the Data Access Layer package in watch mode

• start:embed - Serves the embed script via HTTP, which is required to open the widget

• start:widget - Starts the widget project that shows the widget inside an iframe

• start:queue-manager - Starts the Queue manager application that processes sending data via webhook

• start:web - Starts the web application, where you can create manage imports.

You can run pnpm start:<APP> command one by one too, in case you want to run applications separately.

Set up your environment variables

The command npm run setup:project creates default environment variables that are required to run Impler in a development environment. You can change them based on your requirements. These are all the available environment variables:

Running Tests

Writing tests are currently In Progress. You can test packages or apps using appropriate CLI commands:

API

To run the API tests, run the following command:

npm run test:api

Ports used by services when the project spins up

• 3000 - API

• 4701 - Embed

• 3500 - Widget

• 4200 - Web

Running Migrations

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.

To seed the file that gets downloaded as Sample, one need to pass data in showWidget function, which is exported from the useImpler hook.

This function, when provided with your data, allows Impler to automatically populate sample files with the specified information.

Here is how to do it,

import { useImpler } from '@impler/react';

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "...",
    templateId: "...",
    accessToken: "...",
});

function onShowWidgetClick = () => {
    // Fetch or generate your data from a relevant source
    let data = [ /* Your data here */ ];
    showWidget({ data });
}

<button disabled={!isImplerInitiated} onClick={onShowWidgetClick}>
    Import
</button>

Using the `showWidget` Function

1. Import the useImpler Hook: Start by importing the useImpler hook from @impler/react into your project. This hook provides access to essential Impler functions.

2. Initialize Impler: Create an instance of Impler by calling useImpler with the required configuration, including templateId, projectId, and accessToken.

3. Define Your Data: In the onShowWidgetClick function, gather the data you want to prefill into the sample file. This data should match the structure and content you expect in the final imported files.

4. Call showWidget: Pass your data to the showWidget function, which will handle the process of populating the sample files.

5. Downloading sample file with data: Finally, when user clicks on button with text Import, the Import widget gets opened. Where by clicking on Download sample the user can download the excel file with data provided.

Benefits at a Glance

• Efficient Data Import: Simplify the process of importing relational data by seeding sample file.

• Developer-Friendly: Integrate data sources seamlessly into Impler's import workflow.

• User-Focused: Enhance the user experience by providing prepopulated sample files for download.

With the Data Seeding feature in Impler, you can save time and effort by automating the generation of sample files.

This is particularly valuable when importing data with complex relationships, ensuring a smooth and efficient data import experience.

If you have any questions, suggestions, or comments. Feel free to reach out to us over . We’re constantly improving to deliver the best Data Importer for your product, and we value your input.