El crack de 2008 provocado por la voracidad especulativa del capital financiero cuyos beneficios millonarios estaban asegurados en paraísos fiscales, produjo en España la crisis del ladrillo y la banca. Este escenario, permitió al lobby financiero a través de la…

La entrada Contra la privatización del sistema público de pensiones y por los derechos sociales se publicó primero en Confederación Nacional del Trabajo.

El jueves 7 de octubre, a las 19 horas, tendrá lugar en la sede de nuestra fundación la presentación del libro Caminando entre aristas. Pobres, herejes y malditas del medievo. Para la ocasión contaremos con la presencia de su autor, Jordi Maíz, con quien…

La entrada 7 de octubre. Presentación del libro ‘Caminando entre aristas’ se publicó primero en Confederación Nacional del Trabajo.

Recipe to migrate Drupal 7 Organic Groups to Drupal 9 Group kinta kinta Sun, 10/03/2021 - 22:43

Despite there are some guides about migrating from OG to Group, as the well written https://thinktandem.io/blog/2018/03/30/migrating-drupal-7-organic-groups-to-drupal-8-group/ , and other guidelines such as https://www.drupal.org/project/group/issues/2826544 companion of https://gitlab.com/geeks4change/hubs4change/h4c/-/tree/608842e73cc02c45deede42556693a9db2a7fc37/modules_dev/h4c_migrate/config/install or https://www.drupal.org/node/2797845 . It's really adapted to your environment how you gonna carry on.

This is a short and crash guide that we at communia have done to complete the migrations. When Carrying on this migration must be estimated if the effort of writing code, in a form of migrate plugin classes or yml's, is worth enough than cooking csv's using spreadsheet skills. We've done both.

After importing everything from a standard drupal migration from 7 to 8(with what core offers), the groups types must be created, also their group roles and their group content types in a manual way. The site to be recreated is a news portal that takes headlines from several sites, and each blog or external blog is administrated in group, so being a group can be useful to add dynamic members or authors. So in the example I need to create two group types ( for each node enabled as organic group in d7), bundles are (machine)named:

• blog(it was wrongly named previously bloc). I want also to add a role for each group type as admin to add granularity in permissions. So is needed to create the role admin for each type named:
• blog_group-admin for blog group types.

I also marked the automatic role assignment for creators as admin role. It carries on a little of complexity in membership migrations as we must take care that memberships aren't duplicated (hence the plugin created below), as one membership will be created when groups are created on groups migration step, and another one when we import the memberships on memberships migration step.

So we need a simple module to add the migration configurations in yml, and it will aid us to create some extra plugins and hang on some hooks as hook_migrate_prepare_row. The module tree:

. ├── group-content-export.csv.template ├── group-memberships.csv.template ├── migrations │   ├── d7_node_complete_og_to_canal_blog_extern.yml │   ├── d7_node_complete_og_to_canal_proposta_externa.yml │   ├── rename_d7_node_complete_blog_post.yml │   ├── rename_d7_node_complete_podcast.yml │   ├── upgrade_d7_node_og_blog_content.yml │   ├── upgrade_d7_node_og_blog_translation.yml │   ├── upgrade_d7_node_og_blog.yml │   └── upgrade_d7_user_og_blog_memberships.yml ├── og_migrate_group.info.yml ├── og_migrate_group.module ├── README.md └── src └── Plugin └── migrate ├── process    ├── GroupCreator.php    ├── SkipCreator.php    └── SkipOnNotEmpty.php

Go to gitlab.com to get the actual code for each file: https://gitlab.com/communia/og_migrate_group

So now let's yamling.

Groups

As Organic groups usually was assigned to nodes (if that's not the case cook it as you need), you 'll see it will be based in d7_node migration, so d7_node must be migrated before.

migrations/upgrade_d7_node_og_blog.yml:

langcode: en status: true dependencies: { } id: upgrade_d7_node_og_blog class: Drupal\migrate\Plugin\Migration field_plugin_method: null cck_plugin_method: null migration_tags: - 'Drupal 7' migration_group: insert_group label: 'Insert OG Bloc group' source: plugin: d7_node # The node bundle that must be get from source node_type: bloc process: type: plugin: default_value # put blog as group type (it can be simple mapped here but this is to show how default values work) default_value: blog id: nid # The next uid will be the creator, if default role is marked it will be assigned to this user in group realm uid: node_uid label: title path: alias field_description: body # An illustration of how an image in a group could be used, images must be migrated before as usual (in this example it is not using media module). field_imatge_blogextern: plugin: sub_process source: field_imatge_blogextern process: target_id: fid alt: alt title: title width: width height: height # Here an illustration of how a term could be assigned, once again terms must exist before migrating this. field_tipus_bloc: plugin: sub_process source: taxonomy_vocabulary_7 process: target_id: plugin: entity_lookup source: tid value_key: tid bundle_key: vid bundle: vocabulary_7 entity_type: taxonomy_term revision_uid: revision_uid revision_log: log revision_timestamp: timestamp destination: # It's a group plugin: 'entity:group' migration_dependencies: null

Once module is installed this migration can be imported with:

vendor/drush/drush/drush mim upgrade_d7_node_og_blog

and rolled back with:

vendor/drush/drush/drush migrate:rollback upgrade_d7_node_og_blog

if error locks any migration state saying it's currently importing (when is not) it can be fixed with:

drush migrate:reset-status upgrade_d7_node_og_blog

It's common that sometimes a crufty translation exists, in this example we found two extra fields that covered the translation, instead of using entity_translate or node_translate so field_titolbloces means title field in spanish and field_descripciobloces means field_description in spanish . This configuration will update existing groups translating it. Must be noted that group type must be marked as translatable editing the group type settings. migrations/upgrade_d7_node_og_blog_translation.yml:

langcode: en status: true dependencies: { } id: upgrade_d7_node_og_blog_translation class: Drupal\migrate\Plugin\Migration migration_tags: - 'Drupal 7' migration_group: insert_group label: 'Insert OG blog group translation' migration_dependencies: required: - upgrade_d7_node_og_blog source: plugin: d7_node node_type: 'bloc' process: # Will update the previously imported groups id: plugin: migration_lookup source: nid migration: upgrade_d7_node_og_blog # Where can be found the label in spanish label: plugin: get source: field_titolbloces # Where can be found the description in spanish field_description: plugin: get source: field_descripciobloces # default value of langcode langcode: plugin: default_value default_value: es type: plugin: default_value default_value: blog destination: plugin: 'entity:group' translations: true

This migration can be imported with:

vendor/drush/drush/drush mim upgrade_d7_node_og_blog_translation

and rolled back with:

vendor/drush/drush/drush migrate:rollback upgrade_d7_node_og_blog_translation

if error locks any migration state saying it's currently importing (when is not) it can be fixed with:

drush migrate:reset-status upgrade_d7_node_og_blog_translation

Memberships

As Organic groups memberships usually was assigned to users (if that's not the case cook it as you need), you 'll see it will be based in d7_user migration, so d7_user must be migrated before.

I found that here is easier to use csv, than to carry on too much code to take from here and there, so I have created a view of og memberships with data export in source site. View fields are:

• OG membership: Og membership ID (id)
• OG membership: Entity id (uid)
• OG membership: Group ID (gid)
• (Content entity referenced from og_user_node) Content: Identificator (UID) of the author (gid_uid)

View relationships are:

• OG user node target id. (Content entity referenced from og_user_node) : A bridge to the Content entity that is referenced via og_user_node
• OG membership: User from OG membership (REQUIRED)

If you prefer the sql of that view was:

SELECT og_membership.id AS id, og_membership.etid AS og_membership_etid, og_membership.gid AS og_membership_gid, node_og_membership.uid AS node_og_membership_uid, 'membership_export:page' AS view_name FROM {og_membership} og_membership LEFT JOIN {node} node_og_membership ON og_membership.gid = node_og_membership.nid LEFT JOIN {field_data_og_user_node} node_og_membership__field_data_og_user_node ON node_og_membership.nid = node_og_membership__field_data_og_user_node.og_user_node_target_id AND (node_og_membership__field_data_og_user_node.entity_type = 'user' AND node_og_membership__field_data_og_user_node.deleted = '0') LEFT JOIN {users} og_user_node_node ON node_og_membership__field_data_og_user_node.entity_id = og_user_node_node.uid LEFT JOIN {field_data_og_group_ref} node_og_membership__field_data_og_group_ref ON node_og_membership.nid = node_og_membership__field_data_og_group_ref.og_group_ref_target_id AND (node_og_membership__field_data_og_group_ref.entity_type = 'user' AND node_og_membership__field_data_og_group_ref.deleted = '0') LEFT JOIN {users} og_group_ref_node ON node_og_membership__field_data_og_group_ref.entity_id = og_group_ref_node.uid INNER JOIN {users} users_og_membership ON og_membership.etid = users_og_membership.uid AND (og_membership.entity_type = 'user') LEFT JOIN {profile} profile_users ON users_og_membership.uid = profile_users.uid

Anyway it must output what's in group-memberships.csv.template. Here the csv fields

• id The id of the membership(rarely used, but preserved)
• uid, The user id of the membership (so the member).
• gid, The id of the group that user is member.
• gid_uid, The creator of the group (not used in migration as we take it with a dedicated migration plugin).

"id","uid","gid","gid_uid" "46","1","14829","1" "50","4","14814","4" "52","12","14834","1"

And here the migration that carries on:

langcode: en status: true dependencies: { } id: upgrade_d7_user_og_blog_memberships class: Drupal\migrate\Plugin\Migration field_plugin_method: null cck_plugin_method: null migration_tags: - 'Drupal 7 og' migration_group: insert_group label: 'Insert Group Membership' source: plugin: csv # Must be obtained from source with some exporter (e.g. views data export) and placed in this path path: 'public://imports/group-export.csv' header_row_count: 1 ids: - id column_names: - id: Identifier - uid: UserID - gid: Group delimiter: ',' enclosure: '"' process: label: uid group_creator: # TO illustrate a plugin (not useful in the process), that transforms from a group id to the uid of the group author. - plugin: group_creator source: gid # - # plugin: callback # callable: var_dump # Now will check that group is there via previous migration lookup (the first one we did): gid: - plugin: migration_lookup migration: upgrade_d7_node_og_blog no_stub: true source: gid - plugin: skip_on_empty method: row message: 'gid is missing' entity_id: # To illustrate how the previous `group_creator` could be injected here( not useful): #- # plugin: callback # callable: var_dump # source: "@group_creator" # The plugin created to stop creating membership if current row user uid is the same as the group creator, to prevent duplicated members: - plugin: skip_creator user_key: uid gid_key: gid method: row # Finally check if user uid exists (probably if we are in this process point): - plugin: migration_lookup migration: upgrade_d7_user no_stub: true source: uid - plugin: skip_on_empty method: row message: 'uid is missing' # We hard code the role for all the migrated members: group_roles: plugin: default_value default_value: blog-group_admin type: plugin: default_value default_value: blog-group_membership destination: # Memberships are considerated group_content also. plugin: 'entity:group_content' migration_dependencies: optional: - upgrade_d7_node_og_blog - d7_user

Note the use of skip_creator plugin to stop creating membership if current row user uid is the same as the group creator, to prevent duplicated members. The code of src/Plugin/migrate/process/SkipCreator.php is:

<?php namespace Drupal\og_migrate_group\Plugin\migrate\process; use Drupal\migrate\MigrateExecutableInterface; use Drupal\migrate\ProcessPluginBase; use Drupal\migrate\MigrateException; use Drupal\migrate\MigrateSkipRowException; use Drupal\migrate\MigrateSkipProcessException; use Drupal\migrate\Row; /** * Skip if user_key as uid is the creator of group with gid as in gid_key. * * @MigrateProcessPlugin( * id = "skip_creator" * ) * * To skip if creator use the following: * * @code * field_creator: * plugin: skip_creator * user_key: uid * gid_key: gid * method: row * @endcode * */ class SkipCreator extends ProcessPluginBase { /** * Stops processing the current property when user_key as uid is the creator of group with gid as in gid_key. * * @param mixed $value * The input value. * @param \Drupal\migrate\MigrateExecutableInterface $migrate_executable * The migration in which this process is being executed. * @param \Drupal\migrate\Row $row * The row from the source to process. * @param string $destination_property * The destination property currently worked on. This is only used together * with the $row above. * * @return mixed * The input value, $value, if it is not empty. * * @throws \Drupal\migrate\MigrateSkipProcessException * Thrown if the source property is not set and rest of the process should * be skipped. */ public function process($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { $group = \Drupal::entityTypeManager() ->getStorage('group') ->load($row->getSource()[$this->configuration["gid_key"]]); if (!$group) { throw new MigrateException('gid not found.'); } if ($group->getOwner()->id() == $row->getSource()[$this->configuration["user_key"]]) { throw new MigrateSkipRowException("Creator is the same uid, so it exists"); } return $value; } /** * Skips the current row when user_key as uid is the creator of group with gid as in gid_key. * * @param mixed $value * The input value. * @param \Drupal\migrate\MigrateExecutableInterface $migrate_executable * The migration in which this process is being executed. * @param \Drupal\migrate\Row $row * The row from the source to process. * @param string $destination_property * The destination property currently worked on. This is only used together * with the $row above. * * @return mixed * The input value, $value, if it is not empty. * * @throws \Drupal\migrate\MigrateSkipRowException * Thrown if the source property is not set and the row should be skipped, * records with STATUS_IGNORED status in the map. */ public function row($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { $group = \Drupal::entityTypeManager() ->getStorage('group') ->load($row->getSource()[$this->configuration["gid_key"]]); if (!$group) { throw new MigrateException('gid not found.'); } if ($group->getOwner()->id() == $row->getSource()[$this->configuration["user_key"]]) { throw new MigrateSkipRowException("Creator is the same uid, so it exists"); } return $value; } }

So now we have groups and their members. Now we can start thinking about the content.

Content

Here instead of doing everything at once (create nodes and add as content to groups) as in https://thinktandem.io/blog/2018/03/30/migrating-drupal-7-organic-groups-to-drupal-8-group/ , it will be based in a previously migrated node set, as it was did in previous step where from users it created group memberships.

Let's create the group content first setting that content as available in group type settings/content, in this example blog group available content will be proposta, blog_post(as blog entry), podcast_episode(as audio entry).

The next step starting from commit 9b9c731e it's easier than csv import, so it can be used with the plugin d7_og_content .

Via direct sql query

for group content imports (relation maker):

drush mim rename_d7_node_complete_blog,rename_d7_node_complete_podcast drush mim d7_og_blog_content

It will do the same as it's done in next csv import but using sql to previous import as source

Via view csv import

Despite we could create a new plugin which transforms from nid to title or uid, may be useful  to do it in source view group_content csv, so including these properties. Relevant view used to obtain source data will be:

Relations:

• OG membership: Content from OG membership: 'og_membership_related_node'; REQUIRED
• Entity Reference : Referenced Entity: 'og_group_ref_target_id';

Fields:

• id, label: 'og_membership_id'
• (relationship: og_membership_related_node_1)->nid , label:content_nid
• (relationship: og_membership_related_node_1)->type, label: content_bundle
• (relationship: og_membership_related_node_1)->uid, label: content_uid
• (relationship: og_membership_related_node_1)->title, label: content_title
• gid, label: 'gid'
• (relationship:og_group_ref_target_id)->type, label: group_type

(With the filters and sorts adapted to your case, to prevent listing non-migratable content)

This will output

"og_membership_id","content_nid","content_bundle","content_uid","content_title","gid","group_type" "13397","59546","blog","438","Good news","22477","bloc" "13426","77","podcast","6","El sistema universitari","22447","bloc" "13427","86","proposta","16","Estrobaca","22455","bloc" "13428","87","blog","8","ssl","22449","bloc"

Here the yml that will process this csv:

langcode: en status: true dependencies: { } id: upgrade_d7_node_og_blog_content class: null field_plugin_method: null cck_plugin_method: null migration_tags: - CSV migration_group: insert_group label: 'blog Group Content Migration from CSV' source: plugin: csv # Must be obtained from source with some exporter (e.g. views data export) and placed in this path path: 'public://imports/group-content-export.csv' header_row_count: 1 ids: - og_membership_id column_names: - og_membership_id: Identifier - content_nid: NodeID - content_bundle: bundle - content_uid: uid - content_title: title - gid: group - group_type: GroupType delimiter: ',' enclosure: '"' process: label: # TO show the current item to debug #- # plugin: callback # callable: var_dump # source: content_title - plugin: get source: content_title type: plugin: static_map source: content_bundle bypass: false # Customize this to match your values and group content plugins map: proposta: blog-group_node-proposta blog: blog-group_node-blog_post podcast: blog-group_node-podcast_episode default_value: blog-group_node-blog_post gid: gid entity_id: # To skip if element is not found (may be merged during translation # nodes migration fixes) - plugin: get source: content_nid - plugin: entity_lookup entity_type: node value_key: nid - plugin: skip_on_empty method: row uid: content_uid destination: plugin: 'entity:group_content' migration_dependencies: null

And that's all, after that Groups, Memberships and Content will be migrated.

Bonus track - rename machine name of a bundle when migration has been already executed.

First of all we create the bundle giving the desired name, and with the module field_tools we can make the clone of view_modes first and after that the fields clones. Once it is identical, we must rollback the migration of the content a given bundle: drush migrate:rollback d7_node_complete:podcast and then copy from web/sites/default/files/config__secret_directory_that _must_be_uniquely_and_obscured_from_public_found_at_settings/sync/migrate_plus.migration.upgrade_d7_node_complete_podcast.yml to a dedicated custom module in its migrations directory, then modify file setting the destination to desired bundle machine_name, also must be changed the id and filename according to what you are doing(in this case we set to rename_d7_node_complete_podcast), then run drush migrate:rollback d7_node_complete:podcast --verbose && drush mim rename_d7_node_complete_podcast --verbose to import again the content that we want. Any configuration over it must be executed once again, for example url_redirects migrations.

 

 

Bonus track 2 - node feeds in D7 to feeds_feed entity in D8

First create the feed types and add the fields as it was defined in source nodes that were acting as importers, then edit add the migration configuration, two working samples are in migrations:

• d7_node_complete_og_to_canal_blog_extern.yml
• d7_node_complete_og_to_canal_proposta_externa.yml

Both are based on the stock migration configurations that migrateplus has created and exported in config sync directory: upgrade_d7_node_complete_[bundle_of_source] . You can edit to create the feed entity instead of a node. Be careful to rollback previous imported bundle nodes migrations that match (in example it was feed and proposta_feed bundles). Flush plugins cache, and after that drush mim d7_node_complete_node_to_[bundle_of_source] will do the magic. This is the first step, Now you have two options because  feed source url is not easily picked.

One way and recommended as it can be automatizated, is using a new source plugin to fix the feed urls, look at commit  this is a new plugin that will lookup the feed in source d7 feed table and "magically" will provide the url using some config as this:

 

langcode: en status: true dependencies: { } id: d7_feed_sources class: Drupal\migrate\Plugin\Migration field_plugin_method: null cck_plugin_method: null migration_tags: - 'Drupal 7 feeds' - 'resync' migration_group: resync label: 'Drupal 7 Feeds sources' source: plugin: d7_feed process: fid: - plugin: entity_lookup source: title value_key: title entity_type: feeds_feed # - # plugin: callback # callable: var_dump source: source destination: plugin: 'entity:feeds_feed' migration_dependencies: optional: - d7_node_complete_og_to_canal_blog_extern - d7_node_complete_og_to_canal_proposta_externa

 

The other way is using csv in a manual way,  first instead of struggling too much I end up creating a view in source that outputs to csv these fields:

• Content: title (title)
• Feeds source: Source (source)
• Content: type (bundle)
• Content: Nid (nid)

Which csv I copied to public://imports/export-feeds.csv And created the auxiliary migration configuration to import from csv:

langcode: en status: true dependencies: { } id: fix_feeds_sources class: Drupal\migrate\Plugin\Migration field_plugin_method: null cck_plugin_method: null migration_tags: - 'Drupal 7 feeds' migration_group: fix_feeds label: 'Fix Feeds sources' source: plugin: csv path: 'public://imports/export-feeds.csv' header_row_count: 1 ids: - nid column_names: - title: title - source: Source - bundle: bundle - nid: Identifier delimiter: ',' enclosure: '"' process: fid: - plugin: entity_lookup source: title value_key: title entity_type: feeds_feed # - # plugin: callback # callable: var_dump source: source destination: plugin: 'entity:feeds_feed' migration_dependencies: optional: - d7_node_complete_og_to_canal_blog_extern - d7_node_complete_og_to_canal_proposta_externa

then drush mim fix_feeds_sources again. Then create the feed mappings as it was done in source(or modify as you want). And start importing

Tags Drupal migrations develop

Con firma del nuevo convenio provincial de manipulado de frutas y hortalizas de Almería se pone fin a las expectativas creadas tras las jornadas de huelga en el mes de diciembre del año pasado. Una huelga que irritó a una…

La entrada El nuevo convenio provincial de manipulado de frutas y hortalizas de Almería. Un déjâ vu de otras negociaciones se publicó primero en Confederación Nacional del Trabajo.

Las organitzaciones y grupos del territorio valenciano abajo firmantes dan respaldo a la manifestación convocada por ALIENTE (Aliança Energia i Territori/aliente.org) para el 16 de octubre a las 17h en Madrid. En un momento en que el recibo de la…

La entrada Clamor por una transcición energética que beneficie a las personas se publicó primero en Confederación Nacional del Trabajo.