Concepts
Most of our projects relied on importing data from Excel or CSV files in one way or another. These use-cases prompted us to investigate a possible standardized solution. Over the years this has evolved into a set of complex requirements. While we provide a simple import that should allow users to get started immediately, there are many powerful options hidden in this module. This page serves as an overview of general concepts. To get started using it, read the relevant sections of the developer references.
Overview
The Data Import AccPack imports data from Excel or CSV files into the PublishingHubDB or MuaDB. Three components work together:
- StandardImport plugin: reads the import file and maps columns to entity fields based on the ISON XML configuration
- Camunda process: orchestrates the pipeline. It handles file routing, triggers the plugin, and manages success and error paths.
- ISON XML configuration: defines the mapping between file columns and database entities for a specific project and config ID
Trigger Methods
Two trigger methods are available by default. Both use the same plugin and configuration. They differ in how the file enters the system.
- AdminUI upload: a user uploads a file manually through the AdminUI. The upload Camunda process handles routing to success or error folder.
- Hotfolder / timer: a timer Camunda process runs on a schedule, scans a configured folder, and imports any files it finds. Files are moved to a success or error folder after processing.
Supported Data
The plugin reads Excel (.xlsx) and CSV (.csv) files. An Excel file can contain multiple sheets. Per import row, the plugin can create or update:
- Buckets and bucket content (key value, media asset, price, text, metadata, cord)
- Context: country, language, assortment. Language and country can be static or detected from the file name.
CSV files use a comma as the column delimiter by default. If your file uses a different separator such as ;, configure the delimiter attribute in the XML configuration. The default encoding is UTF-8. For files in a different encoding such as ISO-8859-1, set the charSet attribute. See XML Configuration for details.
Data Processing
Before data is written to the database, three optional processing steps can be applied in the XML configuration.
Transformation allows column values to be modified before they are mapped to entity fields. This means the source file can use any format or naming convention without needing to match the target model. A common case is splitting a combined context string like deu_DE into separate language and country values.
Validation lets you check rows or mapped objects before they are written. A row that fails validation is skipped rather than causing the whole import to fail. This keeps the import resilient to bad data in the source file.
Translation separates display labels from data keys. The column header in the source file serves as the key identifier, while a separate sheet in the same file provides human-readable labels per context. This supports multilingual label management without changing the data structure.
Import Modes
Import mode controls what happens to existing data when re-importing into PublishingHubDB.
| Mode | Behaviour |
|---|---|
delete | Deletes all existing bucket data before importing |
insert | Inserts new buckets and content; does not touch existing records |
update | Inserts new buckets; updates existing buckets and replaces their content |
Import mode applies to PublishingHubDB only. When importing into MuaDB, the plugin inserts new records and updates existing ones automatically (Default Value behaviour).
Target Databases
The same plugin and file format support both PublishingHubDB and MuaDB. The connector instance and entity model configuration differ per target.
| Category | PublishingHubDB | MuaDB |
|---|---|---|
| Import Structure | New Excel structure | New Excel structure |
| MuaConnector Configuration | Not used | Must create a MuaConnector instance. Enable returnAllForEmptyContext to support multiple languages. |
| XML Configuration | Import Mode (v1.1.0) and Default Value (v1.2.0) | Default Value (v1.2.0) only |
| Deployment | Same plugin | |
| Connector Instance | Use PublishingHubDBConnector for EntityModel, Buckets, and all Bucket Contents. | Use MuaConnector for EntityModel, Buckets, and all Bucket Contents. |
For entity model setup and connector configuration details, see Installation and Configuration.