Reference → Import — Reference

Import — Reference

The Import surface is the bulk-ingest plane for SGEN. It owns the controlled path by which external content reaches a site — content files prepared elsewhere, exports from a prior CMS, structured data from a spreadsheet, or a packaged bundle from another SGEN site. Anything that enters the system in volume traces back to this surface.

This page is a reference for platform engineers and integrators who need to understand the surface before scripting against it, scoping a migration, or extending the supported source types. Customer-facing migration walkthroughs live in the customer docs set; this page describes the shape of the surface, not the steps to drive it.

Overview

Import lives under the Tools → Import module in SG-Admin. The module renders three primary views — the source-type picker, the field-mapping editor, and the dry-run preview — and exposes a small set of operations for source upload, mapping persistence, dry-run execution, batch commit, and resume-on-failure.

The module is paired by convention with the Export surface; together they form the portability tier. One half ingests records; the other half emits the bundle shape that the ingest half accepts on the receiving end. Engineers reading the SG-Admin source will see the two surfaces share a common bundle definition.

Where it lives in SG-Admin:

Sidebar: SG-Admin → Tools → Import
URL prefix: /sg-admin/import/
View templates: application/views/Admin/Import/

The module accepts four canonical source families — CSV, XML, JSON, and a WordPress-export shape (a dialect of XML that older agencies still ship). Each family has a recognizer that reads the header / root structure and a mapper that aligns source fields to SGEN record fields. The receiver supports custom-field references — a column or element that does not map to a native field can map to a defined custom field instead.

Imports run in a staged shape. A source upload produces a candidate batch — the system holds the parsed rows, applies the active field map, and produces a preview without committing. Only when the operator confirms does the batch commit row-by-row to the target store. Failed rows do not stop the batch; they collect on a failure list with the rejection reason, and the operator can correct and retry only the failures.

┌──────────────────────────────────────────────────────────────────────┐│ SG-Admin → Tools → Import [+ New import] │├──────────────────────────────────────────────────────────────────────┤│ Source type Target Last used Status ││ ────────────────── ────────────── ────────── ──────────────││ CSV Posts 2 days ago ready ││ CSV Users 5 days ago ready ││ XML Pages never ready ││ JSON Custom Objects 1 day ago ready ││ WordPress export Posts + Media 11 days ago ready ││ SGEN bundle Multi-target never ready ││ ││ [⋯ Configure mapping] [⋯ Dry run] [⋯ View last batch] │└──────────────────────────────────────────────────────────────────────┘

Actions

The Import surface exposes the following operations. Each is described by what it does to the data, not by its internal method name.

Upload source

Accepts the source file (CSV, XML, JSON, or WordPress XML) and detects its shape. Returns a parsed-row count, a column or element list, and the recognized source family. The upload itself does not commit anything to the target store; it produces a staged batch the operator can preview or discard.

The surface enforces a per-source size cap (configurable under Settings → Tools). Sources above the cap return a structured rejection with the cap value and the observed file size. Large migrations split into multiple staged batches.

Configure field mapping

Opens the mapping editor for a staged batch or a saved mapping template. Each source column or element appears as a left-side row; the right side offers the candidate target fields (native fields on the target record type, plus configured custom fields). The operator chooses one target per source field, leaves a source field unmapped (the data is discarded for that field), or marks a source field as the key used to detect duplicates on the target side.

A mapping persists as a named template. Future imports against the same source family can pick the template and skip the editor. Templates are scoped to the target record type — a mapping for "CSV → Posts" does not appear in the picker for "CSV → Users".

Dry-run preview

Applies the active field map to the staged batch without writing to the target store. Returns a preview of the first 25 records as the target shape, the count of rows the system would create, update, skip (duplicates), and reject (validation failures). Rejected rows return with the field that triggered the rejection — required-field missing, format mismatch, unique-key collision, or custom-rule failure.

The dry run is the gate before commit. A batch with rejected rows can still commit — the rejections drop into the failure list. The operator may also choose to fix the source, re-upload, and re-run the dry run.

Commit batch

Writes the staged batch to the target store. The system iterates rows in source order, applies validation, applies the field map, and writes each accepted row. Failures do not stop the batch; they accumulate on the batch's failure list with the row index and rejection reason.

The commit emits progress to the surface — record count processed, current row, accepted, rejected — so a long batch can be monitored. On completion the surface returns the batch summary and a per-status breakdown.

Resume failed rows

After a commit, the operator can open the failure list, correct the source values inline, and re-submit only the failed rows. The system re-applies validation, accepts what passes, and updates the batch summary. Failures that cannot be corrected inline (the source field itself is wrong) can be exported as a CSV for offline correction and a fresh dry-run cycle.

Save mapping template

Persists the active field map as a named template. Templates are reusable across batches, sharable between operators (subject to permission), and editable in place.

List batches

Returns the recent import batches, paginated, with target, source type, row counts (accepted / rejected), operator, and timestamp columns. Each row opens the batch detail view — the failure list, the commit summary, and the mapping template used. Batches are kept for the retention period configured under Settings → Tools.

Cancel staged batch

Discards a staged batch before commit. The source file is removed from the staging store; no target records are affected. A committed batch cannot be cancelled — to reverse a commit, use the standard delete operations on the target record type.

Data model

A batch record carries the following fields. Field names below are the conceptual shape — the on-disk column names match closely but are not contractually stable across releases.

Field	Type	Notes
`id`	integer	Primary key. Stable across edits.
`source_family`	enum	`csv`, `xml`, `json`, `wordpress_export`, `sgen_bundle`.
`target_type`	string	The target record type — `post`, `page`, `user`, `media`, `custom_object`, etc.
`source_filename`	string	Original filename of the upload.
`source_size_bytes`	integer	File size on upload.
`row_count_total`	integer	Total parsed rows.
`row_count_accepted`	integer	Rows committed to the target store.
`row_count_rejected`	integer	Rows on the failure list.
`row_count_skipped`	integer	Rows skipped as duplicates of an existing target record.
`mapping_template_id`	integer	Foreign key to the mapping template used (nullable).
`mapping_snapshot`	JSON	The field map at commit time, carried inline for historical accuracy.
`status`	enum	`staged`, `dry_run`, `committing`, `committed`, `failed`, `cancelled`.
`operator_id`	integer	The user who initiated the batch.
`created_at`	timestamp	Set on upload, immutable.
`committed_at`	timestamp	Set on commit completion.
`failure_list`	JSON	Array of `{row_index, source_values, rejection_reason}` entries.

A mapping template carries:

Field	Type	Notes
`id`	integer	Primary key.
`name`	string	Display name.
`source_family`	enum	Same set as the batch field.
`target_type`	string	Same set as the batch field.
`field_map`	JSON	Map of source field → target field.
`duplicate_key_field`	string	The source field used to detect duplicates (nullable).
`created_at`	timestamp	Immutable.
`updated_at`	timestamp	Touched on save.

Mapping-snapshot semantics: the batch carries the mapping inline as mapping_snapshot so the historical record remains accurate even if the template is later edited or deleted. The mapping_template_id reference is a convenience for finding the template by lineage; the snapshot is the contract.

Duplicate detection: when duplicate_key_field is set on the active mapping, the system checks the target store for an existing record with the same value on the matched target field. The match outcomes are: no match (create), single match (update or skip depending on the batch option), multiple matches (rejected — ambiguous key).

SOURCE UPLOAD│▼┌──────────────────────────┐│ Parse + recognize │ CSV header / XML root / JSON shape└──────────────┬───────────┘│▼┌──────────────────────────┐│ Staged batch │ rows held, no target writes yet└──────────────┬───────────┘│▼┌──────────────────────────┐│ Field mapping │ source field → target field│ + duplicate key │└──────────────┬───────────┘│▼┌──────────────────────────┐│ Dry-run preview │ shows would-create / would-update /│ │ would-skip / would-reject counts└──────────────┬───────────┘│ confirm▼┌──────────────────────────┐│ Commit batch │ iterates rows, writes accepted,│ │ collects rejected on failure list└──────────────┬───────────┘│▼batch summary+ failure list (if any)+ resume-failed-rows path

Permissions

Access to the Import surface is gated at two layers.

Layer 1 — admin gate. Every action under SG-Admin passes through the platform's standard admin access check at request entry. An unauthenticated request never reaches the Import surface.

Layer 2 — per-action capability. Within SG-Admin, each Import action checks a capability associated with the calling operator's role. The default role configuration ships with three roles — Administrator, Editor, Viewer — and the capability map is:

Capability	Administrator	Editor	Viewer
List batches	✔	✔	—
Upload source	✔	✔	—
Configure field mapping	✔	✔	—
Save mapping template	✔	✔	—
Dry-run preview	✔	✔	—
Commit batch	✔	—	—
Resume failed rows	✔	—	—
Cancel staged batch	✔	✔	—
Delete mapping template	✔	—	—

The Administrator role is the only role with commit authority by default. Custom roles defined under Settings → Roles override the map. Operators with upload-but-not-commit capability can prepare batches and hand off to an administrator for the final commit.

Target-record permission cascade. Beyond Import's own gates, each row a batch writes also passes the create or update gate on the target record type. A batch targeting Posts is rejected at commit time for an operator without create-post capability, even if the operator passed the Import-tier checks. The capability cascade is intentional — Import does not grant a back-door to target-record writes.

Audit. Every staged-batch creation, mapping template save or delete, dry-run execution, commit, resume, and cancellation emits an Activity Log entry. The log records the acting operator, the batch identifier, the target record type, and the row-count outcome. The source file itself is retained in the staging store for the retention period configured under Settings → Tools and then purged.

Sensitive-source handling. Sources that carry customer-identifying data (email, phone, address) inherit the privacy posture of the target record type. If the target is a record type with a configured "do not store" field, the import surface honors the redaction at commit time — the source value passes through validation but the target record receives a redacted placeholder, the same way the form-submission surface handles sensitive form fields.

COMMIT REQUEST│▼┌───────────────────────────┐│ Admin gate │ unauth → reject│ (every /sg-admin/* call) │└─────────────┬─────────────┘│ authed▼┌───────────────────────────┐│ Import capability │ no commit cap → reject│ (commit batch) │└─────────────┬─────────────┘│ allowed▼┌───────────────────────────┐│ Target-record cap │ per-row: must hold create / update│ (cascade) │ on the target type└─────────────┬─────────────┘│ row by row▼write or reject│▼batch summary + Activity Log entry

Related references

Export — Reference. Emits the SGEN-bundle shape that this surface ingests on the receiving end. Round-trip portability is defined by the two together.
Settings — Reference. Owns the per-source size cap, batch retention period, default duplicate-handling option, and the Activity Log retention.
Users — Reference. The Import surface targets Users when the source family is CSV → Users; the target-record permission cascade routes through here.
Posts — Reference. Most WordPress-export imports land here. Field-map defaults for post title, body, slug, and category are tuned to the WordPress shape.
Pages — Reference. Same — page imports route through the standard page-record validation.
Media — Reference. WordPress-export bundles include attachments; the Import surface unpacks attachments into the Media library and rewrites post body references to the new media URLs.
Custom Fields — Reference. Mapping templates can route source fields to custom-field definitions when no native target field fits.
Tools — Reference. Import is one tile under the Tools module — sibling tiles include Export, Activity Log, and the database utilities.

For the corresponding customer-facing walkthrough — preparing a CSV, picking a mapping, running a dry run, recovering from failures — see the Import section of the customer docs at /docs/import.

On this page

Guides

Reference

Changelog

Roadmap