Is a pre-built data import library worth the investment?
You have built your own CSV importer, but as your organization, your product and its requirements grow, you ask yourself if switching from your homegrown solution to a pre-built library could be beneficial.
As you grow and onboard more and more clients, your importer increases in complexity and chances are, you can’t afford to keep up with the cost, talent, and time it’ll take to build and maintain your own CSV import solution. Data import cases will increase in complexity, requirements especially performance requirements will rise accordingly and at some point, a data importer without smart features just won’t be able to do the job successfully on a long-term basis.
In general, deciding on build vs buy is crucial, to begin with:
The focus of your developer resources should always be on the product’s core features. The implementation time and the maintenance of secondary/support functionalities such as importer, login, sign up, etc. shall be reduced as far as possible.
However, even for companies that decided to build a CSV importer in-house at some point, replacing the homegrown importer at a later point is worth considering.
Here are some signs that should get you thinking about switching:
In general, software libraries that offer 24/7 support with a quick response time (24h), are well maintained, and are frequently updated (especially in regards to major version releases of common frontend frameworks) can be a valuable solution to replace your homegrown importer.
To provide some numbers and a potential guideline that can support in deciding to make or buy, we created some scenarios covering different levels of import complexities:
Deciding to buy instead of build or replace a homegrown component that already took internal resources should always be driven by a positive ROI calculation considering the investment for setup but also long-term maintenance of the solution.
As the import cases and their complexity can vary significantly, it is important to distinguish and consider different use cases separately.
To provide some reference data and support decision making we will look at four different import scenarios, including their requirements, and discuss their pros and cons:
We consider a basic and straightforward data import case as a first case. The data is imported by internal teams only, the target data model is rather basic, containing around 5–10 columns, no complex validations, and the column mapping is conducted using a simple dropdown view without automation via AI or any other algorithm.
This case meets the minimum requirements, usually has no specific requirements regarding styling or UI, and is implemented quickly.
Experienced software engineers estimate around 2 months of development time, requiring a team of 3–4 engineers working on it.
The simplicity that allows the fast implementation of the importer, unfortunately, comes with some drawbacks that need to be addressed as they will significantly affect the use and manual effort related to data imports:
The second scenario we consider is a more advanced data importer that can be provided to non-technical, external uses as for customer self-service imports.
To ensure a smooth experience for your clients, the advanced CSV importer should fulfill some additional requirements and offer features such as:
The efforts to develop a CSV importer as described above can be estimated at around 3 months of development time for an Engineering team of 3–4 developers.
Even though this more advanced CSV importer can enable self-service imports for your clients and takes manual efforts from internal teams due to smart features and data validation and cleaning functions, however, there are drawbacks that should be considered making a decision of in-house development of this advanced importer:
The third scenario we are looking at is an AI-assisted, high-performance importer optimized for large data, characterized by a simple and advanced UI and includes AI-assisted smart features for automated data mapping, validation, and cleaning.
Being able to provide a CSV import solution equipped with all features mentioned above will not only provide a seamless data onboarding experience to your clients but will also gain significant time savings for your internal teams.
The development time of importers on this level can be estimated at around 8 months (considering a development team of 3–4 developers) and more to be added, as additional features will be required with a growing customer base and increased import complexity over time.
Implementing a plug-and-play import library is considered the fourth scenario in this article. The nuvo Importer is a component that is embedded in the application’s front-end using one of the following frontend frameworks: React, Angular, Vue.js, and plain JavaScript.
The component enables you to guide your users through our data onboarding workflow, including uploading and selecting a file, choosing the preferred sheet and the header row, matching the imported columns to the columns of your target data model, and cleaning the imported data.
Getting started usually takes a couple of minutes only and provides you with a data importer offering:
To conclude, make vs buy or maintaining homegrown vs replacing with an off-the-shelf solution is a decision that has to be considered carefully and depends on the requirements and resources available.
As both variables can develop and vary over time, reconsidering a decision that was once made, can often be worth the investment.
.png)
