Here is the step-by-step guide to embed CSV Excel Importer into your angular application using @impler/angular.

Add Script

You copy this snippet to your code before the closing body tag.

<script type="text/javascript" src="https://embed.impler.io/embed.umd.min.js" async></script>

It will add impler variable in window, so you can call its init and show method.

Install the Package

Add @impler/angular in your application by running the following command.

npm i @impler/angular
# or
yarn add @impler/angular

Use Impler Service

@impler/angular provides ImplerService that you can use to initialize and show an importer to your application.

import { RouterOutlet } from '@angular/router';
import { isPlatformBrowser } from '@angular/common';
import { Component, Inject, PLATFORM_ID } from '@angular/core';
import { EventCalls, EventTypesEnum, ImplerService } from '@impler/angular';

@Component({
  selector: 'app-root',
  standalone: true,
  imports: [RouterOutlet],
  templateUrl: './app.component.html',
  styleUrl: './app.component.css',
})
export class AppComponent {
  title = 'impler-app';

  constructor(
    private implerService: ImplerService,
    @Inject(PLATFORM_ID) private platformId: Object
  ) {
    if (isPlatformBrowser(platformId)) {
      this.implerService.initializeImpler();
    }
  }
  public show(): void {
    this.implerService.showWidget({
      colorScheme: 'dark',
      projectId: '...',
      templateId: '...',
      accessToken: '...',
    });
  }
}

Customize Texts

public show(): void {
  this.implerService.showWidget({
    ...
    texts: {
      STEPPER_TITLES: {
        REVIEW_DATA: 'Check Data', // New Title
      },
    }
  });
}

Listening for Events

Usage Example

...
import { EventCalls, EventTypesEnum, ImplerService } from '@impler/angular';

@Component({
  ...
})
export class AppComponent {
  title = 'impler-app';

  constructor(
    private implerService: ImplerService,
    @Inject(PLATFORM_ID) private platformId: Object
  ) {
    if (isPlatformBrowser(platformId)) {
      this.implerService.initializeImpler();
      this.implerService.subscribeToWidgetEvents((eventData: EventCalls) => {
        switch (eventData.type) {
          case EventTypesEnum.DATA_IMPORTED:
            console.log('Data Imported', eventData.value);
            break;
          default:
            console.log(eventData);
            break;
        }
      });
    }
  }
}

Applying App's Color Scheme

Your application may be developed by keeping dark and light modes in mind. So to let the Import widget show the widget overlay properly, you need to pass colorScheme in showWidget function.

Allowed values for colorScheme are light or dark.

public show(): void {
  this.implerService.showWidget({
    ...
    colorScheme: 'dark',
  });
}

Data Seeding in Sample File

You can provide default data to fill in the Sample file that gets generated from the Import widget. Default data acts as a placeholder for the user to add or update the data further.

Here is an example of how to provide data to fill in the sample file,

public show(): void {
    this.implerService.showWidget({
      data: [
          { country: 'Canada' },
          { country: 'Australia' },
          { country: 'Germany' },
      ]
    });
}

Providing Runtime Schema

The schema and output provided in the web.impler.io portal for any import behave as default values for any Import. It's possible to override the default schema and output, to adapt a dynamic nature of Import.

Here is an example of how to provide dynamic schema and output,

public show(): void {
  this.implerService.showWidget({
      schema: [
          {
            key: 'country',
            name: 'Country',
            type: 'String'
          }
      ],
      output: {
        "%data%": {
          "country_id": "{{country}}"
        },
        "page": "{{page}}",
        "chunkSize": "{{chunkSize}}",
        "isInvalidRecords": "{{isInvalidRecords}}",
        "template": "{{template}}",
        "uploadId": "{{uploadId}}",
        "fileName": "{{fileName}}",
        "extra": "{{extra}}"
      }
  });
}

Advanced Validations

You can provide Advanced Validations in the column as validations key,

public show(): void {
  this.implerService.showWidget({
    schema: [
      {
        "key": "Department Code",
        "name": "Department Code",
        "type": "String",
        "validations": [
          {
            "validate": "unique_with",
            "uniqueKey": "Employee No"
          }
        ]
      },
      {
        "key": "Employee Id",
        "name": "Employee Id",
        "type": "Number",
        "validations": [
          {
            "validate": "unique_with",
            "uniqueKey": "Employee No"
          }
        ]
      }
    ]
  });
}

Using Typescript

If you're using typescript, you can leverage typescript types, like,

import { useImpler, ColumnTypes, ValidationTypes } from '@impler/angular';

public show(): void {
  this.implerService.showWidget({
    schema: [
        {
          key: 'country',
          name: 'Country',
          type: ColumnTypes.STRING,
          "validations": [
            {
              "validate": ValidationTypes.LENGTH,
              "min": 5,
              "max": 100,
              "errorMessage": "Country Name must be between 5 to 100 characters"
            }
          ]
        }
    ]
  });
}

Passing Extra Parameters

It's an obvious need that we want to pass parameters like userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

public show(): void {
  this.implerService.showWidget({
    extra: {
      userId: '4ddhodw3',
      time: new Date().this string()
    }
  });
}

The extra parameters get sent to an application during the webhook call.

Programmatically Closing Import Widget

Impler's import widget provides closeWidget a method that closes the import widget. Useful to close the import widget once data is imported.

public close(): void {
    this.implerService.closeWidget();
}

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

public show(): void {
    this.implerService.showWidget({
        title: "Employee Import"
    });
}

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

public show(): void {
    this.implerService.showWidget({
        primaryColor: '#5f45ff'
    });
}

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side. You can provide a string or async function that returns the token value.

public show(): void {
    this.implerService.showWidget({
        authHeaderValue: async () => {
            return "..."
        }
    });
}

Why?

The ability to import data is often needed in the application. It usually starts the same, reading .csv or .xlsx file and insert records into the database. But after a while, you'll find yourself looping over large files, validating rows, and providing support for file types that you've never heard of before.

Impler's goal is to help developers create an efficient and smooth data import experience between the product and its users. All with an easy-to-use API and outstanding developer experience.

Integrated into your Product

Integrating the import widget into your product can take just a few minutes. The feature-rich experience of the widget makes the data onboarding experience fun for users. Users can map columns and even review invalid records in their spreadsheets so that the right data gets imported.

As developers, we tend to care about the performance of the application, but following the growing number of file types support, mapping, and data validation become a burden on the team and make life hard.

Open Source

One of the reasons for making Impler open-source was to ensure privacy and give back to the community. We heard from many teams and we also felt How hard it's to give support for Data import functionality. So to make it accessible to everyone, we decided to build the Impler with awesome people like you.

Let's do a quick review of how to use @impler/react in your application.

Add Script

You copy this snippet to your code before the closing body tag.

<script type="text/javascript" src="https://embed.impler.io/embed.umd.min.js" async></script>

It will add impler variable in window, so you can call its init and show method.

Install the Package

Add @impler/react in your application by running the following command.

npm i @impler/react
# or
yarn add @impler/react

Add Import Button

@impler/react provides headless useImpler hook that you can use to show an import widget in your application.

import { useImpler } from '@impler/react';

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
});

<button disabled={!isImplerInitiated} onClick={showWidget}>
    Import
</button>

Customize Texts

import { useImpler } from '@impler/react';

const { showWidget, isImplerInitiated } = useImpler({
    ...
    texts: {
      STEPPER_TITLES: {
          REVIEW_DATA: 'Check Data', // New Title
      },
    },
});

<button disabled={!isImplerInitiated} onClick={showWidget}>
    Import
</button>

Listening for Events

Usage Example

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    onUploadStart: (uploadInfo) => {
        console.log("User Started Importing", uploadInfo);
    },
    onUploadTerminate: (uploadInfo) => {
        console.log("User left Import in middle", uploadInfo);
    },
    onUploadComplete: (uploadInfo) => {
        console.log("User completed import", uploadInfo);
    },
    onWidgetClose: () => {
        console.log("Import widget is closed");
    },
    onDataImported: (data) => {
        console.log("Imported data:", data);
    }
});

Applying App's Color Scheme

Your application may be developed by keeping dark and light modes in mind. So to let the Import widget show the widget overlay properly, you need to pass colorScheme in showWidget function.

Allowed values for colorScheme are light or dark.

showWidget({ colorScheme: 'light' })

Data Seeding in Sample File

You can provide default data to fill in the Sample file that gets generated from the Import widget. Default data act as a placeholder for the user to further add or update the data.

Here is an example of how to provide data to fill in the sample file,

showWidget({
  data: [
      { country: 'ABC' },
      { country: 'DEF' },
      { country: 'GHE' },
  ]
});

Providing Runtime Schema

The schema and output provided in the web.impler.io portal for any import behave as default values for any Import. It's possible to override the default schema and output, to adapt dynamic nature of Import.

Here is an example of how to provide dynamic schema and output,

showWidget({
  schema: [
      {
        key: 'country',
        name: 'Country',
        type: 'String'
      }
  ],
  output: {
    "%data%": {
      "country_id": "{{country}}"
    },
    "page": "{{page}}",
    "chunkSize": "{{chunkSize}}",
    "isInvalidRecords": "{{isInvalidRecords}}",
    "template": "{{template}}",
    "uploadId": "{{uploadId}}",
    "fileName": "{{fileName}}",
    "extra": "{{extra}}"
  }
});

Advanced Validations

You can provide Advanced Validations in the column as validations key,

showWidget({
  schema: [
    {
      "key": "Department Code",
      "name": "Department Code",
      "type": "String",
      "validations": [
        {
          "validate": "unique_with",
          "uniqueKey": "Employee No"
        }
      ]
    },
    {
      "key": "Employee Id",
      "name": "Employee Id",
      "type": "Number",
      "validations": [
        {
          "validate": "unique_with",
          "uniqueKey": "Employee No"
        }
      ]
    }
  ]
});

Using Typescript

If you're using typescript, you can leverage typescript types, like,

import { useImpler, ColumnTypes, ValidationTypes } from '@impler/react';

const { showWidget } = useImpler({ ... })

showWidget({
  schema: [
      {
        key: 'country',
        name: 'Country',
        type: ColumnTypes.STRING,
        "validations": [
          {
            "validate": ValidationTypes.LENGTH,
            "min": 5,
            "max": 100,
            "errorMessage": "Country Name must be between 5 to 100 characters"
          }
        ]
      }
  ]
});

Passing Extra Parameters

It's an obvious need that we want to pass userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    extra: {
        userId: '4ddhodw3',
        time: new Date().this string()
    }
});

The extra parameters get sent to an application during the webhook call.

Programmatically Closing Import Widget

Impler's import widget provides closeWidget a method that closes the import widget. Useful to close the import widget once data is imported.

const { showWidget, closeWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    onDataImported: (data) => {
        console.log("Imported data:", data);
        closeWidget();
    }
});

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

Here is how,

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    title: "Employee Import"
});

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    primaryColor: '#5f45ff'
});

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side. You can provide a string or async function that returns the token value.

const { showWidget, isImplerInitiated } = useImpler({
    projectId: "",
    templateId: "",
    accessToken: "",
    authHeaderValue: async () => {
        return "..."
    }
});

1. Setting Bubble App

i. Setting up data type

Prepare a data type in your Bubble app where you want to push the CSV data. Ensure that the data type is Publicly visible. Add custom fields to the data type as per your requirements.

ii. API Settings

2. Go to the API tab

3. Click on Enable Data API

4. Activate API for the data type where you want to push the imported data

5. Generate the API Private Key and save it.

2. Setting up the Impler Application

Visit web.impler.io and log in.

i. Click on the "Create Import" to create the new import

ii. Give name and click on "Create & Continue"

iii. Enable Bubble.io destination

Go to the destination tab and enable Bubble.io. Fill in the following information,

• Bubble App Name - This is the name of your Bubble.io app, in most cases it's available in a URL like https://bubble.io/page?name=index&id=impler-app&tab=tabs-1, in which impler-app is the id.

• Environment - Pick from a development or production environment, based on which environment your app is running on.

  • Custom Domain Name - If the environment is production, provide a domain name address like example.com if you have attached any custom domain to your application.

• API Private Key - It is the API token that you have generated while configuring the ii. API Settings Bubble app.

• Data Type - The name of the data type where data will be sent.

Once done click on Test and Save, which will check details with bubble.io and save the data.

iv. Map Columns

Click on Map Columns to automatically map fields from bubble data type to Impler.

3. Using the Plugin

The last step is to add a CSV Excel Import plugin to your application.

i. Install plugin

1. Go to Plugins and click on Add Plugins

2. Search for CSV Excel Import Plugin

3. Click on Install

ii. Using the Plugin

Now go to the Design panel (1) and search for CSV Excel Importer (2) which is available in the Visual Elements Section. Drag and drop Importer on the UI panel. It doesn't have any visuals so we can hide (3) it.

a. Initialize Importer on Page Load

Go to workflow panel (1). Click on Add Event and add Page Load event (2). For the added event add Initialize Import Widget action, which is available in Element Actions (3).

b. Add a Button on the Page and Add Workflow

From the design section add Button (1), which will open its properties from there click on Add Workflow.

c. Add Workflow to Open Importer on Button Click

From workflow panel (1), for Button (2) add the Open Import Widget action.

d. Configure Importer

Project Id, Template Id, and Access Token are found in the Snippet section of the application.

Once done CSV & Excel Import functionality is ready in your application.

4. Considering UserId while importing data

i. Configuring import plugin to consider UserId

The impler import plugin provides a field for User Id, where one can provide static text or dynamic value as User ID. It's optional. Very useful while adding a relationship with the user.

ii. Passing userId to Bubble.io when data is imported

If you're taking static or dynamic userId field, you have to append "user": {{extra.userId}} in output schema.

Here user is the field name in Bubble.io. You need to follow the name you have in your Bubble.io application.

5. Theming Importer

Provide Primary Color in properties to change the color theme of Importer.

6. Configuring multiple Importers on Page

In Progress

Add Script

You copy this snippet to your code before the closing body tag.

It will add impler variable in window, so we can call its init and show methods later.

Add Import Button

Import widgets get opened when the user clicks on Import button. So let's add an import button,

Initialize Widget

Before the widget gets shown you have to call its init method. So write this code, after added embed script.

Here is the description of what we just did,

1. generateUuid the function generates UUID which helps add multiple import widgets on the same page.

2. init method on impler initializes impler widget.

3. readyCheckInterval calls impler isReady() method in the interval of 1 second to check whether the impler widget is initiated or not.

4. Once Impler is ready we remove disabled attribute from the import button, which the user can click to open the import widget.

Show Widget

After the initialization, when the user clicks on the Import button we will call show method to open the widget,

That's it, your app is now ready to onboard user data into your application.

Customize Texts

Listening to Events

Impler instance provides message an event listener which gets called for various events happening with the widget. Here is how to attach event listener,

eventData follows the { type, value } structure.

Complete Code Example

You can provide default data to fill in the Sample Excel file that gets generated from the Import widget. Default data act as a placeholder for the user to further add or update the data in the file.

Here is an example of how to provide data to fill in the sample file,

Advanced Validations

Here is an example of how to provide dynamic schema and output,

Passing Extra Parameters

It's an obvious need that we want to pass userId or orgId representing who imported the data. In that case, you can pass that data in an extra parameter.

You can pass string, object, or array in extra. Here is an example of how to do it,

The extra parameters get sent to an application during the webhook call.

Changing Import Title

By default, the Import widget takes the name of the Import to show in the title. But it's possible to change the title from the implementation side too. Helpful to keep the title separate from what the name is given in the web portal.

Here is how,

Programmatically Closing the Import Widget

You can close the import widget programmatically by calling close method on impler instance,

Here is how,

Changing Theme Color

You can pass a primary color to the import widget, which will update the colors of all buttons accordingly to match your app's theme color.

Providing Authentication Header Value

In case the backend endpoint is authenticated with the token, it's possible to provide token value from the implementation side.

Impler is built around configuring your import once and letting the user import spreadsheet data every time using Widget.

5 Easy Steps

1. Log in to your Account:

Head over to Impler by visiting https://web.impler.io and log in using your credentials

2. Create a New Import:

After logging in, you will land on the dashboard. Click on Create Import which will open the create modal asking you to give the appropriate name to the Import.

3. Add Columns to Your Import:

Once you've created a new import, you need to add columns to your import spreadsheet. The columns represent the data fields you wish to import.

The image represents a few fields and a button to add a new field.

4. Provide API Destination (Optional):

You can send Imported data to your backend application or Bubble.io application.

Enable Webhook destination and provide a REST api endpoint in case you want to send imported data to your application.

5. Import the data:

It's time to import your CSV or Excel file. We can do it in two ways:

1. Import from Dashboard: You can click on Import the button and the import widget will be opened. Which allows you to use data import functionality without any extra configuration.

2. Embed Widget into Application: You can embed CSV Excel Import widget into your application too, regardless of which tech stack you're using. Follow the steps mentioned in Snippet section of Importer, to embed a widget into your application. Read more in Importer

And that's it! With these five straightforward steps, you are well on your way to revolutionizing your data onboarding process and delivering an exceptional user experience.

Text customization is possible from the application using Customize Texts (React) and Customize Texts (HTML & JS). With this powerful feature, you can update every text on the Import widget. Useful for updating text on import widgets or even implementing localization.

Available Texts

Although you may not need to update every text on the import widget. Here is the complete text object which is used in Impler.

Texts contain variables, Like {total} which is used to put dynamic value. For example, 12K in the case of import confirmation.

{
  COMMON: {
    SORRY: 'Sorry!',
    CANCEL: 'Cancel',
    CLOSE_WIDGET: 'Close',
    UPLOAD_AGAIN: 'Upload Again',
  },
  STEPPER_TITLES: {
    GENERATE_TEMPLATE: 'Generate Template',
    UPLOAD_FILE: 'Upload',
    MAP_COLUMNS: 'Map Columns',
    REVIEW_DATA: 'Review',
    COMPLETE_IMPORT: 'Complete',
    CONFIGURE_JOB: 'Configure',
    SCHEDULE_JOB: 'Schedule',
    CONFIRM_JOB: 'Confirm',
  },
  FILE_DROP_AREA: {
    DROP_FILE: 'Drop and drop a file here or ',
    BROWSE_FILE: 'Browse from computer',
    IMAGE_FILE_SIZE:
      'Image size should be less than 5 MB. Supported formats are PNG, JPG and JPEG.',
    BRING_FILE: 'Bring any .csv or .xlsx file here to start Import',
    FILE_SELECTED: 'File selected successfully',
  },
  'PHASE0-1': {
    IMPORT_FILE: 'Import File',
    GENERATE_TEMPLATE: 'Generate Template',
    IMAGE_INFO_TITLE: 'Generate template with images',
    IMAGE_INFO_SUBTITLE:
      'Drag and drop images below for image columns and generate a template file containing names of uploaded images.',
  },
  PHASE1: {
    SELECT_TEMPLATE_NAME: 'Template',
    SELECT_TEMPLATE_NAME_PLACEHOLDER: 'Select Template',
    SELECT_TEMPLATE_REQUIRED_MSG: 'Please select a template from the list',

    SELECT_SHEET_NAME: 'Select sheet to Import',
    SELECT_SHEET_NAME_PLACEHOLDER: 'Select Excel sheet',
    SELECT_SHEET_CONFIRM: 'Select',
    SELECT_SHEET_REQUIRED_MSG: 'Please select sheet from the list',

    DOWNLOAD_SAMPLE: 'Download sample',
    GENERATE_TEMPLATE: 'Generate Template',
    SEE_MAPPING: 'See Mapping',

    SELECT_FILE_NAME: 'Select a file',
    SELECT_FILE_REQUIRED_MSG: 'Please select a file',
    SELECT_FILE_FORMAT_MSG:
      'File type not supported! Please select a .csv or .xlsx file.',

    TEMPLATE_NOT_FOUND_MSG:
      "We couldn't find the template you're importing! Please check the passed parameters.",
    INCOMPLETE_TEMPLATE_MSG:
      'This import does not have any columns. Please try again after some time!',
  },
  PHASE2: {
    REVIEW_DATA: 'Review Data',
    IN_SCHEMA_TITLE: 'Column in schema',
    IN_SHEET_TITLE: 'Column in your sheet',
    MAPPING_NOT_DONE_TEXT: 'Not Mapped',
    MAPPING_DONE_TEXT: 'Mapping Successful',
    MAPPING_FIELD_PLACEHOLDER: 'Select Field',
  },
  PHASE3: {
    COMPLETE: 'Complete',
    EXPORT_DATA: 'Export Data',
    RE_REVIEW_DATA: 'Re-Review Data',
    ALL_RECORDS_VALID_TITLE: ' All records are found valid!',
    ALL_RECORDS_VALID_DETAILS:
      'All {total} row(s) found valid! Would you like to complete the Import?',
    LABEL_ALL_RECORDS: `All {records}`,
    LABEL_VALID_RECORDS: `Valid {records}`,
    LABEL_INVALID_RECORDS: `Invalid {records}`,

    REPLACE: 'Replace',
    FIND_REPLACE: 'Find and Replace',
    ALL_COLUMNS_LABEL: 'All Columns',

    FIND_LABEL: 'Find',
    FIND_PLACEHOLDER: 'Empty Cell',

    REPLACE_LABEL: 'Replace',
    IN_COLUMN_LABEL: 'In Column',
    CASE_SENSITIVE_LABEL: 'Case Sensitive',
    MATCH_ENTIRE_LABEL: 'Match Entire Cell',
  },
  PHASE4: {
    TITLE: 'Bravo! {count} rows have been uploaded',
    SUB_TITLE:
      '{count} rows have been uploaded successfully and currently is in process, it will be ready shortly.',
    UPLOAD_AGAIN: 'Upload New File',
  },
  AUTOIMPORT_PHASE1: {
    MAPCOLUMN: 'Map Column',
  },
  AUTOIMPORT_PHASE2: {
    SCHEDULE: 'Schedule',
    IN_SCHEMA_TITLE: 'Column in schema',
    IN_FEED_TITLE: 'Key in your RSS feed ',
  },
  AUTOIMPORT_PHASE3: {
    CONFIRM: 'Confirm',
    INVALID_CRON_MESSAGE:
      'Expression values are incorrect. Please update values as per valid values below!',
  },
  DELETE_RECORDS_CONFIRMATION: {
    TITLE: `{total} rows will be deleted. Are you sure?`,
    DETAILS: 'This action cannot be undone.',
    CONFIRM_DELETE: 'Yes',
    CANCEL_DELETE: 'Cancel',
  },
  CLOSE_CONFIRMATION: {
    TITLE: `Are you sure? You will lose your work in progress.`,
    DETAILS: 'Your import is in progress, clicking <b>Yes</b> will reset it.',
    CONFIRM_CLOSE: 'Yes',
    CANCEL_CLOSE: 'No',
  },
}

Texts for Manual Import

When doing manual import of data the most commonly used texts are mentioned below. You can customize the same texts if you want to change language.

{
  "COMMON": {
    "SORRY": "Sorry!",
    "CANCEL": "Cancel",
    "CLOSE_WIDGET": "Close",
    "UPLOAD_AGAIN": "Upload Again"
  },
  "STEPPER_TITLES": {
    "GENERATE_TEMPLATE": "Generate Template",
    "UPLOAD_FILE": "Upload",
    "MAP_COLUMNS": "Map Columns",
    "REVIEW_DATA": "Review",
    "COMPLETE_IMPORT": "Complete",
    "CONFIGURE_JOB": "Configure",
    "SCHEDULE_JOB": "Schedule",
    "CONFIRM_JOB": "Confirm"
  },
  "FILE_DROP_AREA": {
    "DROP_FILE": "Drop and drop a file here or ",
    "BROWSE_FILE": "Browse from computer",
    "IMAGE_FILE_SIZE": "Image size should be less than 5 MB. Supported formats are PNG, JPG and JPEG.",
    "BRING_FILE": "Bring any .csv or .xlsx file here to start Import",
    "FILE_SELECTED": "File selected successfully"
  },
  "PHASE1": {
    "SELECT_TEMPLATE_NAME": "Template",
    "SELECT_TEMPLATE_NAME_PLACEHOLDER": "Select Template",
    "SELECT_TEMPLATE_REQUIRED_MSG": "Please select a template from the list",
    "SELECT_SHEET_NAME": "Select sheet to Import",
    "SELECT_SHEET_NAME_PLACEHOLDER": "Select Excel sheet",
    "SELECT_SHEET_CONFIRM": "Select",
    "SELECT_SHEET_REQUIRED_MSG": "Please select sheet from the list",
    "DOWNLOAD_SAMPLE": "Download sample",
    "GENERATE_TEMPLATE": "Generate Template",
    "SEE_MAPPING": "See Mapping",
    "SELECT_FILE_NAME": "Select a file",
    "SELECT_FILE_REQUIRED_MSG": "Please select a file",
    "SELECT_FILE_FORMAT_MSG": "File type not supported! Please select a .csv or .xlsx file.",
    "TEMPLATE_NOT_FOUND_MSG": "We couldn't find the template you're importing! Please check the passed parameters.",
    "INCOMPLETE_TEMPLATE_MSG": "This import does not have any columns. Please try again after some time!"
  },
  "PHASE2": {
    "REVIEW_DATA": "Review Data",
    "IN_SCHEMA_TITLE": "Column in schema",
    "IN_SHEET_TITLE": "Column in your sheet",
    "MAPPING_NOT_DONE_TEXT": "Not Mapped",
    "MAPPING_DONE_TEXT": "Mapping Successful",
    "MAPPING_FIELD_PLACEHOLDER": "Select Field"
  },
  "PHASE3": {
    "COMPLETE": "Complete",
    "EXPORT_DATA": "Export Data",
    "RE_REVIEW_DATA": "Re-Review Data",
    "ALL_RECORDS_VALID_TITLE": " All records are found valid!",
    "ALL_RECORDS_VALID_DETAILS": "All {total} row(s) found valid! Would you like to complete the Import?",
    "LABEL_ALL_RECORDS": "All {records}",
    "LABEL_VALID_RECORDS": "Valid {records}",
    "LABEL_INVALID_RECORDS": "Invalid {records}",
    "REPLACE": "Replace",
    "FIND_REPLACE": "Find and Replace",
    "ALL_COLUMNS_LABEL": "All Columns",
    "FIND_LABEL": "Find",
    "FIND_PLACEHOLDER": "Empty Cell",
    "REPLACE_LABEL": "Replace",
    "IN_COLUMN_LABEL": "In Column",
    "CASE_SENSITIVE_LABEL": "Case Sensitive",
    "MATCH_ENTIRE_LABEL": "Match Entire Cell"
  },
  "PHASE4": {
    "TITLE": "Bravo! {count} rows have been uploaded",
    "SUB_TITLE": "{count} rows have been uploaded successfully and currently is in process, it will be ready shortly.",
    "UPLOAD_AGAIN": "Upload New File"
  },
  "DELETE_RECORDS_CONFIRMATION": {
    "TITLE": "{total} rows will be deleted. Are you sure?",
    "DETAILS": "This action cannot be undone.",
    "CONFIRM_DELETE": "Yes",
    "CANCEL_DELETE": "Cancel"
  },
  "CLOSE_CONFIRMATION": {
    "TITLE": "Are you sure? You will lose your work in progress.",
    "DETAILS": "Your import is in progress, clicking <b>Yes</b> will reset it.",
    "CONFIRM_CLOSE": "Yes",
    "CANCEL_CLOSE": "No"
  }
}

Normal Flow

The normal importer flow has a higher number of steps to import the file and follows the below structure,

1. Download Sample File.

2. Write data into the file using spreadsheet editing software like Excel, Libre Office Calc, Numbers, etc.

3. Upload the file in which data is entered.

4. Select the header row.

5. Map columns.

6. Review data and fix invalid data if there is any.

7. Complete the Import.

This process takes a lot of time and involves many steps in the data import process. We introduced the facility to "Directly enter your data" to reduce the time and effort required to complete the import.

Directly enter your data

• Open the Importer and click on Directly enter your data.
• The spreadsheet editor will be opened which will show the column names as header and as per column types, columns will be shown. For example, for the date column, a date picker will be shown.
• Data will be validated instantly, so you can just continue editing the data and the importer will continue validating it in the background.

• While pasting the data directly from the spreadsheet, the validation will be performed after pasting is done.

• The user can not Finish Import if the data contains invalid values.

How it works?

• Open the Importer and select the file you want to import.
• Select Header step will be opened. Which shows the first 15 rows for header selection. You can select the Header row of your choice or continue importing without headers by clicking on File does not have headers.
• Mapping will be based on the selected header row. Rows above the header row will not be imported.
• In the case of importing files without headers, the first row will be shown in the Mapping Step.

Validations

Range

Validates that field's values for Number and Double columns fall within a min and max values. Either min, max or both must be defined. Values are inclusive.

Length

Validates that field's value for String column falls within min and max number of characters.

Either min, max or both must be defined. Values are inclusive.

Unique Across Multiple Fields

Validate uniqueness across multiple fields. To use it, place a unique_with validation on each field that you want to be part of the uniqueness validation, all sharing the same uniqueKey.

Here is an example set of fields using unique_with validations used to validate that the combination of department code and employee Id fields are unique.

Here is the example of unique_with validation in action,

Adding Validations

From Platform

1. Click on Validations while adding column or click on Edit to edit the column.
2. Assign validations as needed,

From Code

Once the user completes importing the spreadsheet in the widget, the impler sends imported data to the callback URL mentioned in the destination.

If the callback URL is protected, there is a facility to add header-based authentication. To activate it you can mention authHeaderName in the destination section and its value to the import button as authHeaderValue prop.

How to add Webhook Destination?

1. Open the Import and go to the Destination section.

2. Enable Webhook.

3. Provide REST API Endpoint of your application.

Chunk Size

You can provide Chunk Size too, which defines how many records Impler will send you in one request.

Authentication

Impler will call your API endpoint from the server, so it will not be visible to anyone. For additional security, you can provide Auth Header Name which will be the key of headers sent, while the value will be taken from the application when import starts.

Details to provide Auth Header Value is available at Providing Authentication Header Value for react and at Providing Authentication Header Value for HTML & JS Embed.

Webhook Data Format

List of available validators:

1. String Validator: This validator ensures that the column value is a string. String and Number are both valid values.

2. Number Validator: The Number validator verifies that the column value is a valid numeric entry. It only permits numerical values like 12 but will not allow 12.33 and john.

3. Double Validator: The Double validator verifies that the column value is either a Number or a Number with decimals. Valid values are 12 and 12.5 but john is not valid.

4. Email Validator: With the Email validator, you can ensure that the column value conforms to a valid email format. Valid values look like [email protected] while values like john and john.com are not valid.

5. Regex Validator: The Regex validator enables you to define a custom regular expression pattern that the column value must match.

6. Select Validator: Use the Select validator to restrict column values to a predefined set of options. For instance, you can use it to ensure that a "Gender" column contains only values like "Male" or "Female."

7. Any Validator: Any validator offers maximum flexibility by allowing any value in the column.

Validator Enhancements

• isRequired: Ensures that a value must exist in each cell for the column.

• isUnique: Ensures that the value is unique throughout the column.

• allowMultiSelect: Accepts multiple values separated by , for the cell.

It's possible to extend validation functionality to adjust according to your needs. Read more in Custom Validation.

Once the user completes importing the spreadsheet in the widget, you access imported data immediately on the frontend application.

Enable Frontend Callback Destination

1. Open the Import and go to the Destination section.

2. Enable Get Imported Data in Frontend.

Get Imported Data on Frontend

Once the destination is enabled Impler will start sending Imported data on the frontend.

To access we can use onDataImported method at Listening for Events if using react or can check for aDATA_IMPORTED event as per Listening to Events if using HTML & JavaScript.

Users filling excel sheet often get's confused with "What value will go into this column?", "In which format I need to add phone number?", "What format to follow for date?", etc. We can answer these user questions with Column Descriptions.

How to provide column description?

Description input is available in Column Details Form, which get's opened by clicking on Validations Button.

Or by clicking on Edit Column button.

Which opens column modal which has input for column description.

How column description works?

Provided column description gets visible in review step while showing imported data to user.

It's possible to change border of tooltip by providing primaryColor while opening import widget.

How do I enable the automated import feature?

The process of automatic data import looks the same as manual data import, where the user uploads a file and impler imports the data. To convert any import to automated import we just need to switch its mode from Manual to Automatic. Once done your import is transformed from Manual to Automatic Import Mode.

How does Automated Import work?

When we switch import mode to Automatic, the Import widget gets transformed and instead of accepting the file from the user, the import widget asks the user for a source from where data will get imported on interval.

At the moment impler accepts RSS URL as a source from where data will be fetched at user-specified intervals. The embed process of automatic import is the same as manual import. Here is how automatic import works:

Step 1: Configure

The first step is to provide a source from where data will be imported at regular intervals. Here on RSS URL input, the user can provide the RSS URL and Impler will keep importing data from the URL.

Step 2: Map Columns

The second step is to Map appropriate columns from the RSS file. As RSS follows XML structure, Impler parses the RSS file, extracts all paths for possible values in the XML file, and returns headings. Users can pick an appropriate heading for the column.

Step 3: Schedule Timing

Impler imports data from specified sources at regular intervals. In the schedule step user defines the Interval. The ability to specify intervals in Minutes, Hours, Days, Months, and Days makes it limitless for a user to provide any timing.

Step 4: Confirm

At the end, impler confirms the automated import configuration. Which shows the source and when it will run.

Automated Imports are associated with users

You can provide anything in externalUserId. ID or Email address that uniquely identifies the user is best suitable for value in externalUserId.

Showing Automated Imports in Application

Automated Imports refers to the user. It becomes obvious to show users import jobs into your application. Here are all the APIs to deal with import jobs in Impler.

Getting User Import Jobs

Pause the User Import Job

Resume the User Import Job

Delete the User Import Job

To implement custom validation logic, you can use the code editor provided in the Validator tab of the Import Details page.

Your custom validation function should adhere to this structure:

exports.code = async (params) => {
    let errorRecords = [];
    // Your custom validation logic here
    return errorRecords;
}

The code function must have the name code, and it can return an array of error records if validation failures occur. Impler will call this code function when performing custom validation.

Parameters provided to the `code` Function

The params object provided to your custom validation function contains essential information about the import process and the data being validated. It follows this structure:

{
    uploadId: string;
    extra: string | number | json;
    fileName: string;
    data: [
        {
            index: number;
            record: {
                [key: string]: string | number;
            };
            errors: {};
            isValid: boolean;
        }
    ];
    totalRecords: number;
    chunkSize: number;
}

• uploadId is the id of import happening at the moment.

• extra is the string, number or json provided at the time of import.

• fileName is the name of file that is being imported.

• data is the array of records, where each record follows this strcuture:

  • index of the record

  • record the actual record object being imported

  • errors is an key-value pair of errors for key of record if there is any

  • isValid flag indicating whether current record has errors or not

• totalRecords number indicating totalRecords being imported

• chunkSize number indicating current size of records

Returned Error Records

Error records returned from your custom validation code should follow this structure:

[
    { 
        index: 1, 
        errors: {
            email: 'Email is already in use.'
        } 
    }
]

In the case of all records passing validation, return an empty array [].

Example, Validating data against database data

Here's an example of how to validate data against data from a database using the Axios library:

const axios = require('axios');

exports.code = async (params) => {
  let errorRecords = [];
  let users = await axios.get('https://your-api.com/users');
  let userEmails = users.map(user => user.email);
  
  for(let item of params.data) {
    let error = {
      index: item.index,
      errors: {},
    }
    if(userEmails.includes(item.record.email)) {
      error.errors['email'] = "Email must be unique."
    }
    errorRecords.push(error);
  }
  return errorRecords;
}

Features

Access to variables inside code editor

Developers can access schema keys by pressing Ctrl + Space to display a list of available variables and selecting from it.

Additionally, the suggestion box contains variables available in params, such as params.uploadId, params.fileName, and more.

Performing HTTP calls

Your custom validation code can make HTTP calls using libraries like Axios. For example, you can retrieve data from your own API and validate it against the imported data.

Validation execution in chunks

The custom code function is executed in chunks, typically in batches of 500 records.

For example, if you are validating 10,000 records, the function will be called 20 times to validate the data in manageable segments.

Combining with static validation

Error records generated from your custom validation code are merged with errors generated from Impler's static validation processes, creating a comprehensive error report.

Scalable Architecture

Impler's custom validation functionality is built on a scalable architecture, allowing you to handle and validate large datasets, even in the millions.

With custom validation in Impler, you have the flexibility to implement intricate validation logic, ensuring data integrity and quality in your import processes.

This feature empowers you to customize your imports to meet your specific needs, making complex data imports more accessible and efficient.

How do I enable Image upload in Import?

The first step for image import is to add or update a column with the Image type. In case providing Runtime Schema keep column type as Image and importer will be ready to import images.

How does Image Import work?

Once the import is configured to import data with Image. We can start importing data with Images. Here is the image import flow,

1. Generate a Template with Images,

The first step is to upload all images and generate a template with uploaded image names. All columns with the type Image will appear in Select Column dropdown and Uploaded images will be shown below Upload Image section.

Once all image files are selected, click on Generate Template to generate an Excel file holding the names of all uploaded images.

2. Upload Excel with data and image names.

Clicking on Generate Template will download an Excel file and Importer will move to the Upload section. You can enter data in the downloaded Excel file and select image names for image columns.

It's possible to generate a template again by clicking on Generate Template.

Once selected click on Map Columns.

3. Do mapping for columns

Click on Review Data to verify data getting imported.

4. Review Data

The review step checks if image columns contain the name of the uploaded image. If not shows a validation error.

Click on Re-Review Data to complete the import, once all records are valid.

5. Complete Import

6. Get Imported Data

Imported data will be sent to a webhook with an endpoint to download and access the uploaded file.

7. Download or Access the uploaded file

The download file endpoint is secured using accesskey which is available in Settings section on the left side. Here is the CURL for a review,

curl --location --request GET 'https://api.impler.io/v1/upload/66b759d087a854c817cd3e98/asset/logo-icon.png' \
--header 'accesskey: <ACCESS_TOKEN>'

Till the date, if imported spreadsheet contains missing value or any column is not mapped, the response uses {{key}} as placeholder. By utilizing Default Value functionality developer has more control over format of response they will receive.

Setting Default Values

Default value can be applied to columns either at the time of adding column or updating column. Wildcard values like null, undefined can also be used, which available in the dropdown.

Means developer can choose one of these predefined values or write default value as a string of their choice.

1. null: Represents the null value.

2. undefined: Represents an undefined value.

3. Empty String: Represents an empty string.

4. Empty Array ([]): Represents an empty array.

5. Boolean true: Represents the boolean value true.

6. Boolean false: Represents the boolean value false.

While providing custom schema you can mention with defaultValue key and put your desired string or appropriate value from mentioned table.

When default value gets applied?

The provided default value is used when imported data gets sent to application as response. The default value will be applied to column when,

1. Column Not Required and Not Selected During Mapping:

   • When a column is not marked as required, and the user chooses not to select the column during the mapping phase, the default value will be used in response.

2. Column Not Required and Contains Empty Value:

   • When a column is not marked as required, and the column contains an empty value, the default value will be applied to fill the gap.

Providing default value during Runtime Schema

Web portal GUI makes it easy to assign default value to column. Based on needs you can provide default value while providing schema during runtime too.

Default Value example with Runtime Schema

// Custom Schema
const schema = {
  columns: [
    { name: "description", required: false, defaultValue: "<<undefined>>" },
    // Other columns...
  ],
};

The Default Value functionality provides flexibility and control to developers when importing data. Whether dealing with optional columns or handling empty values, this feature enhances the overall data import experience.

Purpose

Consider you have an ERP application that needs to import sales information. Our system provides a way to add extra information to sales information, like challan-no. Other users might have different columns.

In that case, fixing the import data structure is impossible. So we need to pass schema during run time when the user clicks on Import button to import the data.

Example Schema

Available properties for schema

Here are the keys

• name (required): The name of the column, will be displayed in the import widget during mapping.

• key (required): The key of the column, will be used to create an Excel file and match it with available headings in the file.

• type (optional): One of the types from String, Number, Date, Email, Regex, Select, and Any to override the existing type for the column.

• isRequired (optional): A boolean value indicating whether values in the current column are required.

• isUnique (optional): A boolean value indicating whether values in the current column must be unique.

• selectValues (optional): Select values indicating what are the possible values when the type is select.

• regex (optional): A regular expression to override when the type is regex.

• dateFormats (optional): Date formats indicating a list of formats acceptable for the date field. Required when type is Date. For example, ['dd/mm/yyyy','dd/mm/yy']

To seed the file that gets downloaded as Sample, one need to pass data in showWidget function, which is exported from the useImpler hook.

This function, when provided with your data, allows Impler to automatically populate sample files with the specified information.

Here is how to do it,

Using the `showWidget` Function

1. Import the useImpler Hook: Start by importing the useImpler hook from @impler/react into your project. This hook provides access to essential Impler functions.

2. Initialize Impler: Create an instance of Impler by calling useImpler with the required configuration, including templateId, projectId, and accessToken.

3. Define Your Data: In the onShowWidgetClick function, gather the data you want to prefill into the sample file. This data should match the structure and content you expect in the final imported files.

4. Call showWidget: Pass your data to the showWidget function, which will handle the process of populating the sample files.

5. Downloading sample file with data: Finally, when user clicks on button with text Import, the Import widget gets opened. Where by clicking on Download sample the user can download the excel file with data provided.

Benefits at a Glance

• Efficient Data Import: Simplify the process of importing relational data by seeding sample file.

• Developer-Friendly: Integrate data sources seamlessly into Impler's import workflow.

• User-Focused: Enhance the user experience by providing prepopulated sample files for download.

With the Data Seeding feature in Impler, you can save time and effort by automating the generation of sample files.

This is particularly valuable when importing data with complex relationships, ensuring a smooth and efficient data import experience.

There are chances that the system which is receiving data is not made or managed by us. But as a plugin or integration, we need to send imported data to that system. In that case, Impler allows changing the output format which follows the system's syntax.

Output Format Sample

{
  "%data%": {
    "month": "{{month}}",
    "day": "{{day}}",
    "AverageTemperatureFahr": "{{AverageTemperatureFahr}}",
    "AverageTemperatureUncertaintyFahr": "{{AverageTemperatureUncertaintyFahr}}",
    "city": "{{city}}",
    "country_id": "{{country_id}}",
    "Country": "{{Country}}",
    "Latitude": "{{Latitude}}",
    "Longitude": "{{Longitude}}"
  },
  "page": "{{page}}",
  "chunkSize": "{{chunkSize}}",
  "isInvalidRecords": "{{isInvalidRecords}}",
  "template": "{{template}}",
  "uploadId": "{{uploadId}}",
  "fileName": "{{fileName}}",
  "extra": "{{extra}}"
}

Usage Examples

Joining First Name and Last Name

{
  "%data%": {
    "FullName": "{{firstName}} {{lastName}}"
  },
  "page": "{{page}}",
  "chunkSize": "{{chunkSize}}",
  "isInvalidRecords": "{{isInvalidRecords}}",
  "template": "{{template}}",
  "uploadId": "{{uploadId}}",
  "fileName": "{{fileName}}",
  "extra": "{{extra}}"
}

Providing an Object for each value

{
  "%data%": {
    "city": {
      "label": "City",
      "id": "city",
      "value": "{{city}}"
    }
  }
}

Passing an Array for value

{
  "%data%": {
    "address": ["{{addressLine1}}","{{addressLine2}}", "{{state}}", "{{city}}", "{{pincode}}"]
  }
}

Output Format works differently for dynamic schema

If you're providing schema runtime from the application, in that case, default provided output isn't taken into consideration. Because columns defined in the output might not match with columns getting imported.

In that case, we need to provide schema runtime. We have listed examples for react in Providing Runtime Schema and for HTML & JS versions in Providing Runtime Schema.

This output customization capability empowers you to direct imported data to any destination and in any preferred format.

Three ways to freeze columns

We can freeze columns in many ways. Here is how,

i. Freeze the column while adding a new one

1. Click on the + button to add a new column.

2. Provide column Name, Key name and rest of the values based on your choice.

3. Check Frozen? checkbox for the column.

ii. Freezing already created column

1. Click on the Pencil (Edit) icon for any created column.

2. Check the Frozen? checkbox.

3. Click on Save.

We can freeze the column while providing a custom schema too with the isFrozen flag in our frontend code.

Effects on Sample File Generation

The generated sample file will have specified columns freeze on the left side. So they won't gets hidden if we scroll to the right.

Effects on Spreadsheet Editor

While importing data spreadsheet editor freezes two default Checkbox and Sr. No. Columns. Other mentioned columns get frozen as per schema.

We heard issues like "I imported data but didn't receive callback", "Imported data is not showing in my application", "Validation code is not working as expected", etc. Also, "imported data not seen in the application", is something no one wants to face.

As a solution, Impler will send alert mail to users in the following scenarios,

1. Added webhook endpoint is not proper.

2. Application threw an error while sending imported data over the webhook.

3. Validation code throws an error during execution, which happens when data import is in progress.

Here is the sample mail,

i. Enable Multiselect from web portal

1. Click on the + button to add a new column.

2. Provide column Name and Type = Select.

1. Click on Validations.

1. We can provide allowed values in Select Values.

2. We can also enable Multi Select for select column. Which allows user to select multiple values in the cell.

1. Additionally we can sepcify , or ; as delimiter for multi-select values.

iii. Enabling Multiselect Dropdown using Runtime Schema

We can enable the Multiselect feature by providing a custom schema with the allowMultiSelect flag in our frontend code.

showWidget({
  schema: [
    {
      key: 'choice',
      name: 'Eating Choice',
      type: 'Select',
      selectValues: ['Fruit', 'Vegetables', 'Nuts'],
      allowMultiSelect: true,
      delimiter: ';'
    }
  ]
});

Sample file generation and format

Impler creates .xlsx and .xlsm files based on the columns' configuration:

• .xlsx File: Generated if no columns have Multiselect enabled.

• .xlsm File: Generated if any column has Multiselect enabled.

Writing in .xlsm Files

When a column is marked as Multiselect, Impler appends #MULTI to the end of the column name in the Excel file. This allows users to select multiple options in the cell.

For detailed steps on how to write in .xlsm files refer to Writing effectively into .xlsm files.

Data Format

Columns with the Multiselect feature will send an array of selected values when sent to application Using Webhook or Using Frontend Callback.

The Activity Page provides a comprehensive overview of your data imports, giving you insights into import statistics, trends, and detailed histories.

Let's explore the various sections and functionalities that the Activity Page has to offer.

1. Overview Statistics

The Activity Page greets you with key statistics that provide a quick snapshot of your data import activity:

• Total Imports: The count of all imports conducted using Impler.

• Total Records: The cumulative number of records imported across all your imports.

• Total Error Records: The count of records that encountered errors during the import process.

2. Import Count Trends (Bar Chart)

A graphical representation of the import count over the last seven days is displayed using a bar chart. This visual tool allows you to quickly identify trends and fluctuations in your import activity.

3. Import History Table

The Import History Table presents a paginated view of your import history, facilitating a deeper dive into each import's details. This table consists of the following columns:

• Import Id: A unique identifier for each import.

• Name: The name associated with the import.

• Date: The date on which the import occurred.

• Time: The time at which the import was initiated.

• Status: The status of the import, categorized as Upload, Mapped, Review, Confirmed, or Completed. The "Completed" status indicates that the imported data has been successfully sent to the application.

• Total Records: The total number of records in the import.

Filtering and Insight

The Import History Table offers filtering capabilities, allowing you to narrow down the displayed history by date and import name. This enables you to quickly locate specific imports and track their progress over time.

Import Insights

The Activity Page serves as a valuable resource for gaining insights into your import operations:

• Overview: Get a bird's-eye view of your import statistics.

• Trends: Identify patterns and trends in your import activity with the import count bar chart.

• Detailed History: Dive deep into each import's journey, from initiation to completion.

• Error Analysis: Monitor error records to address data quality issues.

Whether you're seeking a quick overview or a comprehensive analysis, the Activity Page equips you with the tools you need to assess your data import processes effectively.

Impler's Activity Page is designed to streamline your import tracking, optimize your workflow, and empower you with data-driven decision-making capabilities.

Projects in Impler play a vital role in organizing imports effectively. You can invite your peers to the team and collaborate on Imports.

How Projects Work?

Open Projects Modal

Click on your project name on the top left corner of the Impler platform below the Impler logo to open Projects Modal.

Create Project

You can create unlimited projects in Impler. To create a new project write the project name in text input and click on Create. Upon creating a project you will be switched to a new project automatically.

Switch Project

Once the project is created it will be shown on the projects menu. As Owner only you can delete the project.

With inspiration from Birds of a feather flock together, you can invite your peers in Impler with different background, who are responsible for different work. Here are the details about which role has which capabilities in Impler.

Roles and Permissions

How Invitation Flow Works

1) Create a new account or Sign In

2) Click on `Invite` Button on Team Members

The person who created project will be on Admin role by default.

3) Provide Member Details and their Roles

Based on your current plan you can invite members to your team. Provide email IDs of your members and their roles. An invitation email will be sent to the invited members with a link to take action on the invitation.

4) Receive Invitation Link by Email or Copy from Sent Invitations

Invited members will get an email to accept the invitation or you can copy the invitation link from Sent Invitations section on Team Members.

5) Take Action on Invitation

On visiting the invitation link, if anyone is new to Impler, they can create an account or if they already have an account they can simply click on Accept Invitation.

Invitation Link View if a user is not logged In

While clicking on Signup

Email will be automatically filled while going to signup from invitation section.

Take action on the invitation

Impler provides the most generous way to integrate world-class CSV Excel Import functionality in any application. Impler only considers Imported Records as a metric to collect the amount and not additional parameters like no. of imports and no. of templates.

Working of Subscription in Impler

Impler assigns every user the Starter Plan, which includes 5,000 free records per month. Here’s how the subscription and billing process works:

• The Starter Plan Includes 5,000 free records per month. Importing more than 5,000 records incurs a charge relative to $1 for every 10,000 records.

• By default, the Starter Plan renews every month. If you have imported more than 5,000 records, you must pay the outstanding amount to renew your subscription. You can choose an appropriate plan to cover this.

• You must add a card before purchasing any plan. Adding a card creates a mandate of a certain amount, which helps the user to not provide payment details repeatedly and backend to deduct the amount at intervals.

• Plan upgrades take effect immediately. While Plan downgrades occur at the end of the current billing cycle.

• Users can cancel their plan anytime. Upon cancellation, the canceled plan does not renew at the end of the billing cycle, and the default Starter Plan is reactivated. Outstanding amounts are deducted at the end of the billing cycle after cancellation.

Impler is built upon scalable architecture to validate and process records of any size.

Separation of concern helps organizing code across various packages, libraries and apps. The idea is that app is divided into various parts, and each one is responsible for a spacific task.

Let's dive deep into building blocks of Impler.

The mental model

Widget

Widget is the main app where import happens. Widget provides UI to upload, map, validate and reivew data files. Widget gets embedded into iframe using embed script. It communicates with API to accomplish file processing work.

Queue Manager

Queue manager handles processing data. Once user completes file import, the command gets passed to Queue Manager app to start processing data and sending it to application in chunks.

Authentication

API keys are used to authenticate APIs happening from import widget. Additionally developers can integrate Impler into their application by manually calling Impler API.

SDKs

Occasionally, Impler might add features that call for data or database schema alterations. This typically occurs when a feature requires certain data to be present on a database entity to function. To make these modifications to your database, you can utilize migrations.

Running Migrations

To run data migrations, enter the following sequence of commands in your terminal from the impler/api repository root:

npm run setup:project
cd apps/api
npm run migration -- ./src/migrations/<MIGRATION_PATH>.ts
# e.g. npm run migration --  ./src/migrations/update-date-formats/update-date-formats.migration.ts

Certain features can require more than one migration; in that case, you must do each migration in the following order.

Migrations History

Below you will find a list of migrations introduced in previous versions of Impler, alongside the migration path to use in the script above.

Requirements

• Node.js version v18.13.0

• MongoDB

• RabbitMQ

• localstack (required for storing files)

• (Optional) pnpm - Needed if you want to install new packages

You can also run docker containers for MongoDB, RabbitMQ, and Localstatck.

Setup the project

After installing the required services on your machine, you can clone and set your forked version of the project:

• Fork Impler's repository. Clone or download your fork to your local machine.

• Run the initial setup command npm run setup:project to install and build all dependencies.

• Run the project locally using npm run start:dev

The npm run start:dev will start all of the services in parallel including APIs and Widget clients. if you only want to run parts of the platform, you can use the following run commands from the root project.

• start:api - Starts the API in watch mode

• start:dal - Builds the Data Access Layer package in watch mode

• start:embed - Serves the embed script via HTTP, which is required to open the widget

• start:widget - Starts the widget project that shows the widget inside an iframe

• start:queue-manager - Starts the Queue manager application that processes sending data via webhook

• start:web - Starts the web application, where you can create manage imports.

You can run pnpm start:<APP> command one by one too, in case you want to run applications separately.

Set up your environment variables

The command npm run setup:project creates default environment variables that are required to run Impler in a development environment. You can change them based on your requirements. These are all the available environment variables:

Running Tests

Writing tests are currently In Progress. You can test packages or apps using appropriate CLI commands:

API

To run the API tests, run the following command:

npm run test:api

Ports used by services when the project spins up

• 3000 - API

• 4701 - Embed

• 3500 - Widget

• 4200 - Web

Running Migrations

The .xlsm file format includes macro code that enables the Multiselect feature in Impler. Due to varying security protocols on different operating systems, there are specific steps to follow to ensure these macros run effectively.

Windows

Microsoft Excel on Windows restricts macro execution by default due to security concerns. Follow these steps to enable macros and use the Multiselect feature:

Steps to Enable Macros in Excel

1. Download the Sample File:

   1. First, download the sample file generated by Impler. If any column has Multiselect enabled, it will be in .xlsm format.

   2. By default, Microsoft Excel displays an Security Risk error for such files in its latest versions.
2. Mark the File as Safe:

   1. Navigate to the file location.

   2. Right-click on the .xlsm file and select Properties.

   3. In the Properties window, under the Security section, check the Unblock checkbox.

   4. Click Apply.
3. Enable Macros:

   • Open the file in Microsoft Excel.

   • An orange banner will appear at the top, indicating that macros are disabled.

   • Click on Enable Content it to allow the macros to run.

These steps will ensure that the macro code for the Multiselect feature is enabled, allowing you to select multiple options within the specified columns.

Linux

Linux users typically use LibreOffice Calc, which does not run macro code by default. However, you can still work with the Multiselect feature by manually entering options.

Steps for LibreOffice Calc

1. Open the .xlsm File:

   • Open the .xlsm file in LibreOffice Calc.

   • A warning message about macros will appear.
2. Manually Enter Options:

   • Since macros do not run in LibreOffice Calc, you need to manually type in the options for Multiselect.

   • For example, if you are entering countries, you can write New York,India,Canada in the cell.

By following these steps, you can effectively use the Multiselect feature in .xlsm files on both Windows and Linux.

Summary

Windows:

• Download the .xlsm file.

• Mark the file as safe through Properties.

• Enable macros in Excel to use the Multiselect feature.

Linux:

• Open the .xlsm file in LibreOffice Calc.

• Manually type the options in multi-select fields.