Register a new module
The twocream Shopware Connector Bundle offers three ways to implement an import module.
AbstractModule is suitable for simple entities and is used, for example, for attribute imports. It is also used when you want to import flat data structures from the MDM area into your own Shopware entities.
AbstractSyncModule is suitable for more complex entities, such as the Shopware product entity. It provides the option to create, update, or delete multiple entities during processing. For example, relational records like Shopware units of measure.
AbstractTextSyncModule builds upon AbstractSyncModule and is suitable for simple text entities that only consist of a key and translations.
For the import of products, the plugin provides the ProductModule as of version 3.0.0, which is adapted to the project via hooks only, see "Product import".
Structure of a simple module
Simple modules must inherit from this abstract class: Twocream\JsonImporter\Core\System\Import\Module\AbstractModule
<?php
namespace MyCustomExtension\Core\System\Import\Module;
use Shopware\Core\Framework\DataAbstractionLayer\Event\EntityWrittenContainerEvent;
use Twocream\JsonImporter\Core\System\Import\Module\AbstractModule;
use Twocream\JsonImporter\Core\System\Import\Validation\Decision;
use Twocream\JsonImporter\Core\System\Import\Validation\Result;
use MyCustomExtension\Core\Content\Pimcore\MasterDataManagement\Video\VideoDefinition;
class VideoModule extends AbstractModule
{
public function getHandledPackage(): string
{
return 'videos';
}
protected function getEntityName(): string
{
return VideoDefinition::ENTITY_NAME;
}
protected function validateEntry(array $entry): array
{
$problems = [];
if (empty($entry['uuid'])) {
$problems[] = 'uuid';
}
if (empty($entry['id'])) {
$problems[] = 'id';
}
return $problems;
}
protected function store(Result $result): ?EntityWrittenContainerEvent
{
$records = [];
$videoRepository = $this->registry->getRepository('app_pimcore_video');
foreach ($result as $entry) {
if ($entry->getDecision() === Decision::DECISION_SKIP_RECORD) {
continue;
}
$translations = $this->service->prepareTranslation($entry['texts']['translations']);
$videoTranslations = $this->service->prepareTranslation($entry['video']['translations'], [
'youtube-link' => 'youtubeLink'
]);
$thumbnailTranslations = $this->service->prepareTranslation($entry['thumbnail']['translations'], [
'image' => 'thumbnailImage',
'title' => 'thumbnailTitle',
'alt-text' => 'thumbnailAltText'
]);
foreach ($translations as $key => $values) {
if (isset($videoTranslations[$key])) {
$translations[$key] = array_merge($values, $videoTranslations[$key]);
} else {
$translations[$key] = array_merge($values, ['youtubeLink' => null]);
}
if (isset($thumbnailTranslations[$key])) {
$translations[$key] = array_merge($translations[$key], $thumbnailTranslations[$key]);
} else {
$translations[$key] = array_merge(
$translations[$key],
['thumbnailImage' => null, 'thumbnailTitle' => null, 'thumbnailAltText' => null]
);
}
}
$records[] = [
'id' => $entry['uuid'],
'pimcoreId' => $entry['id'],
'key' => $entry['key'],
'type' => 'youtube',
'defaultLink' => $entry['video']['default-value'],
'translations' => $translations
];
}
if (!empty($records)) {
return $videoRepository->upsert($records, $this->context);
}
return null;
}
}
Structure of a more complex module
Complex modules must inherit from this abstract class: Twocream\JsonImporter\Core\System\Import\Module\AbstractSyncModule
The following example shows the basic structure of a complex module using a custom product module. As of version 3.0.0, the ProductModule of the plugin should be extended for the package products instead of building a custom module (see "Product import").
<?php
namespace TwocreamExampleExtension\Core\System\Import\Module;
use Doctrine\DBAL\Connection;
use Shopware\Core\Content\Media\Aggregate\MediaFolder\MediaFolderDefinition;
use Shopware\Core\Content\Product\Aggregate\ProductConfiguratorSetting\ProductConfiguratorSettingDefinition;
use Shopware\Core\Content\Product\Aggregate\ProductMedia\ProductMediaDefinition;
use Shopware\Core\Content\Product\Aggregate\ProductProperty\ProductPropertyDefinition;
use Shopware\Core\Content\Product\Aggregate\ProductVisibility\ProductVisibilityDefinition;
use Shopware\Core\Content\Product\ProductDefinition;
use Shopware\Core\Content\Property\Aggregate\PropertyGroupOption\PropertyGroupOptionDefinition;
use Shopware\Core\Content\Product\DataAbstractionLayer\ProductIndexer;
use Shopware\Core\Framework\Api\Sync\SyncOperation;
use Shopware\Core\Framework\Api\Sync\SyncService;
use Shopware\Core\Framework\DataAbstractionLayer\DefinitionInstanceRegistry;
use Shopware\Core\Framework\DataAbstractionLayer\Event\EntityWrittenContainerEvent;
use Shopware\Core\Framework\DataAbstractionLayer\Search\Criteria;
use Shopware\Core\Framework\DataAbstractionLayer\Search\EntitySearchResult;
use Shopware\Core\System\SalesChannel\SalesChannelDefinition;
use Shopware\Storefront\Framework\Seo\SeoUrlRoute\SeoUrlUpdateListener;
use Symfony\Component\EventDispatcher\EventDispatcherInterface;
use Twocream\JsonImporter\Core\System\Import\Module\AbstractSyncModule;
use Twocream\JsonImporter\Core\System\Import\Module\Service\ModuleService;
use Twocream\JsonImporter\Core\System\Import\Validation\Decision;
use Twocream\JsonImporter\Core\System\Import\Validation\Result;
use TwocreamExampleExtension\Helper\UuidConstants;
use TwocreamExampleExtension\Helper\SalesChannelHelper;
use TwocreamExampleExtension\Service\ExampleProductService;
use TwocreamExampleExtension\Service\ExampleMediaService;
class ProductModule extends AbstractSyncModule
{
private array $variantPropertyGroupOptionPositions = [];
private array $existsOptionIdsInProductConfigurator = [];
private EntitySearchResult $salesChannels;
private string $mediaFolderId;
private ExampleProductService $productService;
private ExampleMediaService $mediaService;
public function __construct(
SyncService $syncService,
EventDispatcherInterface $dispatcher,
Connection $connection,
DefinitionInstanceRegistry $registry,
?ModuleService $moduleService = null,
ExampleProductService $productService,
ExampleMediaService $mediaService,
) {
parent::__construct($syncService, $dispatcher, $connection, $registry, $moduleService);
$this->productService = $productService;
$this->mediaService = $mediaService;
$this->mediaFolderId = md5('pimcore-media-folder');
$this->syncs[MediaFolderDefinition::ENTITY_NAME][SyncOperation::ACTION_UPSERT][] = [
'id' => $this->mediaFolderId,
'name' => 'Pimcore Media',
'useParentConfiguration' => false,
'configuration' => []
];
}
public function getHandledPackage(): string
{
return 'products';
}
protected function getEntityName(): string
{
return ProductDefinition::ENTITY_NAME;
}
protected function validateEntry(array $entry): array
{
$problems = [];
if (empty($entry['translations']['en_GB']['name'])) $problems[] = 'name';
return $problems;
}
protected function store(Result $result): ?EntityWrittenContainerEvent
{
$variantPropertyGroupOptions = $this->registry->getRepository(
PropertyGroupOptionDefinition::ENTITY_NAME
)->search(new Criteria(), $this->context);
$this->salesChannels = $this->registry->getRepository(SalesChannelDefinition::ENTITY_NAME)->search(
(new Criteria())
->addAssociation('countries')
->addAssociation('languages')
->addAssociation('languages.locale'),
$this->context
);
foreach ($variantPropertyGroupOptions as $variantPropertyGroupOption) {
$this->variantPropertyGroupOptionPositions[$variantPropertyGroupOption->getId()] = $variantPropertyGroupOption->getPosition();
}
foreach ($result as $entry) {
if ($entry->getDecision() === Decision::DECISION_SKIP_RECORD) continue;
if ($this->productService->isProductNumberAlreadyExisting($entry['uuid'], $entry['product-number'])) continue;
$variantsIds = [];
foreach ($entry['variants'] as $variant) {
$variantsIds[] = $variant['uuid'];
}
$this->storeProduct($entry->getValue(), null, $variantsIds);
foreach ($entry['variants'] as $variant) {
$this->storeProduct($variant, $entry['uuid']);
}
$this->existsOptionIdsInProductConfigurator = [];
if (empty($variantsIds)) continue;
$this->syncs[ProductDefinition::ENTITY_NAME][SyncOperation::ACTION_UPSERT][] = [
'id' => $entry['uuid'],
'mainVariantId' => $variantsIds[array_key_first($variantsIds)]
];
}
return $this->cleanupEntityWrittenContainerEvent(
$this->sync(),
ProductDefinition::ENTITY_NAME
);
}
// Separate function for products
protected function storeProduct(array $entry, ?string $parentProductId, array $variantsIds = []): void
{
$price = [
'eur' => [
'currencyId' => UuidConstants::CURRENCY_EUR_ID,
'gross' => $entry['price']['gross'],
'net' => $entry['price']['net'],
'linked' => false
]
];
$visibilities = [];
$hasVisibility = false;
foreach ($this->salesChannels as $salesChannel) {
$salesChannelId = $salesChannel->getId();
foreach (SalesChannelHelper::SALES_CHANNEL_VISIBILITY_MAPPING[$salesChannelId] as $country) {
if (!isset($entry['visibilities'][$country])) continue 2;
}
$visibilities[$salesChannelId] = [
'id' => md5($entry['id'] . '-ProductVisibilityDefinition-' . $salesChannelId),
'productId' => $entry['uuid'],
'salesChannelId' => $salesChannelId,
'visibility' => ProductVisibilityDefinition::VISIBILITY_ALL,
];
$hasVisibility = true;
}
// Products should be displayed in INT if they are visible in at least one other sales channel
if (!empty($visibilities) && !isset($visibilities[SalesChannelHelper::SALES_CHANNEL_INT_ID])) {
$visibilities[SalesChannelHelper::SALES_CHANNEL_INT_ID] = [
'id' => md5(
$entry['id']
. '-ProductVisibilityDefinition-'
. SalesChannelHelper::SALES_CHANNEL_INT_ID
),
'productId' => $entry['uuid'],
'salesChannelId' => SalesChannelHelper::SALES_CHANNEL_INT_ID,
'visibility' => ProductVisibilityDefinition::VISIBILITY_ALL,
];
}
$propertyGroupOptionIds = $this->prepareRelations(
$entry['id'],
$entry['properties'],
PropertyGroupOptionDefinition::ENTITY_NAME
);
$variantPropertyGroupOptionId = null;
// The attribute value / PropertyGroupOption for variant display
// must not be imported as a 'property' on the product, but as an 'option'
foreach ($propertyGroupOptionIds as $propertyGroupOptionId => $relationData) {
if (array_key_exists($propertyGroupOptionId, $this->variantPropertyGroupOptionPositions)) {
$variantPropertyGroupOptionId = $propertyGroupOptionId;
unset($propertyGroupOptionIds[$propertyGroupOptionId]);
break;
}
}
$productMedia = $this->mediaService->getProductMedia(
$entry['custom-fields']['twocream-product-connector-media'],
$entry['product-number']
);
if ($this->isIdExist($entry['uuid'], ProductDefinition::ENTITY_NAME)) {
$product = $this->productService->fetchExistingProduct($entry['uuid']);
foreach ($product->getVisibilities() ?? [] as $visibility) {
if (!isset($visibilities[$visibility->getSalesChannelId()])) {
$this->syncs[ProductVisibilityDefinition::ENTITY_NAME][SyncOperation::ACTION_DELETE][] = [
'id' => $visibility->getId(),
'productId' => $product->getId(),
'salesChannelId' => $visibility->getId()
];
} else {
// If the sales channel is already assigned it doesn't need to be assigned again
unset($visibilities[$visibility->getSalesChannelId()]);
}
}
foreach ($product->getProperties() ?? [] as $property) {
if (!in_array($property->getId(), array_keys($propertyGroupOptionIds))) {
$this->syncs[ProductPropertyDefinition::ENTITY_NAME][SyncOperation::ACTION_DELETE][] = [
'productId' => $product->getId(),
'optionId' => $property->getId()
];
}
}
foreach ($product->getConfiguratorSettings() ?? [] as $configuratorSetting) {
$this->syncs[ProductConfiguratorSettingDefinition::ENTITY_NAME][SyncOperation::ACTION_DELETE][] = [
'id' => $configuratorSetting->getId()
];
}
foreach ($product->getChildren() as $variant) {
if (!in_array($variant->getId(), $variantsIds)) {
$this->syncs[ProductDefinition::ENTITY_NAME][SyncOperation::ACTION_DELETE][] = [
'id' => $variant->getId()
];
}
}
$currentMedia = [];
$newMedia = [];
foreach ($product->getMedia() ?? [] as $media) {
$currentMedia[] = $media->getMediaId();
}
foreach ($productMedia ?? [] as $media) {
$newMedia[] = $media['mediaId'];
}
if (!empty(array_diff($newMedia, $currentMedia))) {
foreach ($product->getMedia() ?? [] as $media) {
$this->syncs[ProductMediaDefinition::ENTITY_NAME][SyncOperation::ACTION_DELETE][] = ['id' => $media->getId()];
}
}
}
$translations = $this->service->prepareTranslation(
$entry['translations'],
[
'description' => 'description',
'name' => 'name',
'meta-title' => 'metaTitle',
'meta-description' => 'metaDescription'
]
);
$instructionsForUse = array_map(function ($translation) {
return $translation['twocream_pimcore_export_product_instructions_for_use'] ?? '';
}, $this->service->prepareTranslation(
$entry['custom-fields']['twocream-product-connector-instructions-for-use'],
['instructionsForUse' => 'twocream_pimcore_export_product_instructions_for_use']
));
$sets = array_map(function ($translation) {
return $translation['twocream_pimcore_export_product_sets'] ?? '';
}, $this->service->prepareTranslation(
$entry['custom-fields']['twocream-product-connector-sets'],
['setInformation' => 'twocream_pimcore_export_product_sets']
));
foreach ($translations as $languageId => $translation) {
$translations[$languageId]['customFields']['twocream_pimcore_export_product_instructions_for_use'] =
$instructionsForUse[$languageId];
$translations[$languageId]['customFields']['twocream_pimcore_export_product_sets'] =
json_encode($sets[$languageId]);
$translations[$languageId]['customFields']['twocream_pimcore_export_product_position'] =
$entry['custom-fields']['twocream-product-connector-position'];
$translations[$languageId]['customFields']['twocream_pimcore_export_product_downloads'] =
json_encode($entry['custom-fields']['twocream-product-connector-downloads']);
$translations[$languageId]['customFields']['twocream_pimcore_export_product_icons'] =
json_encode($entry['custom-fields']['twocream-product-connector-icons']);
$translations[$languageId]['customFields']['twocream_pimcore_export_product_videos'] =
json_encode($entry['custom-fields']['twocream-product-connector-videos']);
$translations[$languageId]['customFields']['twocream_pimcore_export_product_distribution_countries'] =
json_encode(array_keys($entry['visibilities']));
}
$productData = [
'id' => $entry['uuid'],
'parentID' => $parentProductId,
'active' => $hasVisibility,
'productNumber' => $entry['product-number'],
'manufacturerId' => $this->productService->fetchManufacturerId($entry['manufacturer']),
'markAsTopseller' => $entry['mark-as-topseller'],
'taxId' => $this->productService->fetchTaxId((int) $entry['tax']),
'price' => array_values($price),
'deliveryTimeId' => null,
'shippingFree' => $entry['shipping-free'],
'minPurchase' => $entry['min-purchase'] ?? 1,
'purchaseSteps' => 1,
'maxPurchase' => null,
'releaseDate' => $entry['release-date'],
'visibilities' => array_values($visibilities),
'ean' => $entry['ean'],
'media' => $productMedia,
'cover' => !empty($productMedia) ? array_shift($productMedia) : null,
'width' => $entry['width'],
'height' => $entry['height'],
'length' => $entry['length'],
'weight' => $entry['weight'],
'purchaseUnit' => empty($entry['purchase-unit']) ? 1 : $entry['purchase-unit'],
'unitId' => $this->productService->fetchUnitId($entry['unit']),
'referenceUnit' => empty($entry['purchase-unit']) ? 1 : $entry['purchase-unit'],
'properties' => array_values($propertyGroupOptionIds),
'translations' => $translations
];
if ($variantPropertyGroupOptionId) $productData['options'][] = ['id' => $variantPropertyGroupOptionId];
$this->syncs[ProductDefinition::ENTITY_NAME][SyncOperation::ACTION_UPSERT][] = $productData;
}
protected function getSkipIndexers(): array
{
return [
ProductIndexer::SEARCH_KEYWORD_UPDATER,
SeoUrlUpdateListener::CATEGORY_SEO_URL_UPDATER,
SeoUrlUpdateListener::PRODUCT_SEO_URL_UPDATER,
];
}
}
Structure of a text module
Text modules must inherit from this abstract class: Twocream\JsonImporter\Core\System\Import\Module\AbstractTextSyncModule
AbstractTextSyncModule already implements validateEntry() and store(). Only getHandledPackage() and getEntityName() have to be implemented.
<?php
namespace MyCustomExtension\Core\System\Import\Module;
use Twocream\JsonImporter\Core\System\Import\Module\AbstractTextSyncModule;
class SnippetModule extends AbstractTextSyncModule
{
public function getHandledPackage(): string
{
return 'snippets';
}
protected function getEntityName(): string
{
return 'my_custom_snippet';
}
}
Each entry of the package is processed as follows:
| JSON | Shopware |
|---|---|
uuid | id |
id | pimcoreId |
key | key |
texts: { <locale>: { ... } } | translations – the keys per language are adopted unchanged |
Entries without uuid, id or key are skipped. If the Pimcore ID of the entity is not stored in the field pimcoreId, the method getSourceIdField(): string can be overridden.
Explanation
The complex module builds upon the simple module, therefore the following definitions apply to both types.
| Name | Description |
|---|---|
getHandledPackage(): string | Unique package name, e.g., "products" |
getEntityName(): string | The name of the entity the module is intended for, e.g., "product" |
store(Result $result): ?EntityWrittenContainerEvent | Here, the content-related processing of data for import occurs |
canBeDeleted(): bool | Optionally, it can prevent data deletion |
isIdExist(string $id, string $entityName): bool | Helper method to check if a record with the given ID & entity already exists |
validateEntry(array $entry): array | Optionally, you can implement validation logic for imported data |
cleanupEntityWrittenContainerEvent(?EntityWrittenContainerEvent $entityWrittenContainerEvent, string $entityName): ?EntityWrittenContainerEvent | Clean up method for complex import structures |
getPackageDependencies(): array | Optional: packages that must be imported before the module's own package, see Dependencies between packages |
prepareRelations(...): array | Helper method to prepare relations like [{id, uuid}] for Shopware, see Preparing relations |
Additionally, the following methods are available in modules extending AbstractSyncModule:
| Name | Description |
|---|---|
getSkipIndexers(): array | This method can be used to suppress certain indexing processes. It should return a list of indexer names |
cleanupRelations(...): void | Helper method to delete relations that are no longer part of the import, see Deleting stale relations |
Preparing relations
The method prepareRelations() of AbstractModule converts a list of relations from the JSON (e.g. properties: [{id, uuid}]) into records for Shopware. Relations that do not exist in Shopware are skipped and reported as a warning. The result uses the UUID of the relation as key.
protected function prepareRelations(
string $contextId,
array $relations,
string $entityName,
bool $includePosition = false,
string $fkFieldName = 'id',
array $additionalData = [],
array $keysToKeep = []
): array
| Parameter | Description |
|---|---|
$contextId | ID of the imported record under which warnings are reported |
$relations | List of relations, e.g. [{id, uuid}]. If the uuid is missing, it is generated via md5(<id>) |
$entityName | Entity of the relation, e.g. property_group_option |
$includePosition | Adds the position of the relation in the list as position |
$fkFieldName | Field name under which the UUID of the relation is stored, e.g. mediaId |
$additionalData | Additional data added to every relation |
$keysToKeep | Keys of a relation that are additionally adopted. The keys are converted to camelCase |
// $entry['media'] = [['id' => 12, 'uuid' => '...', 'is-cover' => true], ...]
$media = $this->prepareRelations(
(string) $entry['id'],
$entry['media'],
MediaDefinition::ENTITY_NAME,
true,
'mediaId',
[],
['is-cover']
);
// Result: ['<uuid>' => ['mediaId' => '<uuid>', 'position' => 0, 'isCover' => true], ...]
As of version 3.0.0, prepareRelations() has the parameter array $keysToKeep = []. Modules that override this method must adjust the signature accordingly.
Deleting stale relations
The method cleanupRelations() of AbstractSyncModule deletes existing relations of a record that are no longer part of the import. The deletion is queued in $this->syncs and executed with the next call of sync().
protected function cleanupRelations(
string $entityName,
string $sourcePropertyField,
string $sourceId,
string $targetStorageField,
array $newRelations
): void
| Parameter | Description |
|---|---|
$entityName | Entity of the relation, e.g. product_media |
$sourcePropertyField | Field of the relation that references the imported record, e.g. productId |
$sourceId | ID of the imported record |
$targetStorageField | Field from which the ID of an existing relation is read if the entity has multiple primary keys |
$newRelations | IDs of the relations that are part of the import |
$this->cleanupRelations(
ProductMediaDefinition::ENTITY_NAME,
'productId',
$entry['uuid'],
'id',
array_column($productMedia, 'id')
);
Unlike the method removeDeprecatedRelations(), which deletes relations via the combination of $sourcePropertyField and a target field (e.g. productId and optionId), cleanupRelations() deletes the relations via their own id.
If a module already has its own method cleanupRelations(), it must use the same signature including the return type void as of version 3.0.0, or be removed.
Dependencies between packages
Before the packages of an import file are processed, they are sorted by the dependencies of their modules. For this purpose, a module declares via getPackageDependencies() the packages that must be imported before its own package:
public function getPackageDependencies(): array
{
return ['products'];
}
The following rules apply to the sorting:
- The sorting is stable: packages are only moved if a dependency requires it, otherwise the order of the import file is kept
- Dependencies that are not part of the import file are ignored
- Cyclic dependencies do not result in an endless loop
The modules of the plugin declare the following dependencies:
| Package | Module | Depends on |
|---|---|---|
attributes | Module\Product\PropertyGroupModule | – |
attribute-values | Module\Product\PropertyGroupOptionModule | attributes |
products | Module\Product\ProductModule | attributes, attribute-values |
cross-selling | Module\Product\CrossSellingModule | products |
category-assignments | Module\Product\CategoryAssignmentModule | products, cross-selling |
The ProductModule is not registered by the plugin (see "Product import").
Afterwards, the event PreHandlePackagesEvent::NAME is dispatched with the sorted packages. It can still be used to adjust the order for the project (see "Events").
Register the module
New modules must be registered in a service configuration file. The service tag twocream_json_importer.module must be used.
For structured configuration, it is recommended to create the file src/Resources/config/services/import.xml.
This can then be included via an import into the central services.xml.
<service id="MyCustomExtension\Core\System\Import\Module\ProductModule">
<tag name="twocream_json_importer.module"/>
</service>
Replace a module of the plugin
A subclass of a module that the plugin registers itself (PropertyGroupModule, PropertyGroupOptionModule, CrossSellingModule, CategoryAssignmentModule) must not additionally be registered with the service tag: the plugin module keeps its tag, JsonFileHandler::findModule() uses the first module of a package and the subclass may be ignored silently. Register the subclass as decorator of the plugin service instead. Symfony moves the tag twocream_json_importer.module from the decorated service to the decorator, so only the subclass handles the package:
<service id="MyCustomExtension\Core\System\Import\Module\Product\CategoryAssignmentModule"
decorates="Twocream\JsonImporter\Core\System\Import\Module\Product\CategoryAssignmentModule"
autowire="true"/>
The decorator gets no own tag and the decorated module is not injected, the subclass replaces it. Redefining the service id of the plugin with another class also works, but only if the project configuration is loaded after the plugin – the decorator does not depend on the load order. The ProductModule is not registered by the plugin itself, a subclass is therefore registered with the tag (see "Registering the product module").