Check cannot find Drush classes and functions

[fgm]

In custom code, I get errors on:

• Function drush_backend_batch_process not found. (in a drush command class)
• Parameter $context of method updateChunk has invalid typehint type DrushBatchContext. This one is in a phpdoc like this:

  /**
   * ...snip...
   * @param \DrushBatchContext|array $context
   *   The batch context.
   */
  public static function updateChunk(
    /* ...snip ... */
    &$context) {

This function is in a service and receives context as an array in the web ui or a DrushBatchContext when used in Drush, hence the variant hint.

This is with a local composer-installed Drush 9.6.2.

[mglaman]

Interesting. So drush/drush:^9 is on the local installation, which means the files should be autoloaded properly. I think phpstan-drupal may need to add autoloading support for Drush if its missing entries in its autoload_files

[fgm]

Yes, composer.json contains this require line: "drush/drush": "^9.6.2",
but I've had to use the standalone global phar version of drupal-check, as I could not install it with Composer locally because of conflicting requirements (SF4.2 for some drupal-check dependencies, SF3.4 only for drush/drupalconsole requirements). So maybe it does not use the local autoloader, which might explain the issue.

[mglaman]

It still loads the Drupal autoloader, otherwise it couldn't detect Drupal classes.

@fgm could you provide the sample command/module path you're checking? I would like to add a CircleCI job for it

[fgm]

Here they are, stripped as much as feasible. Not sure how that can help, though. ISTR the Drush includes (for non-class code like drush_backend_batch_process()) are loaded by Drush itself, not by the autoloader.

drush.services.yml (partial)

services:
  magpjm.commands:
    class: \Drupal\magpjm\Commands\MagpjmCommands
    arguments:
      - '@magpjm.year_setter'
    tags:
      - { name: drush.command }

magpjm.services.yml (partial)

  magpjm.year_setter:
    class: \Drupal\magpjm\YearSetter
    arguments:
      - '@entity_type.manager'
      - '@entity_type.bundle.info'
      - '@entity_field.manager'
      - '@messenger'

src/Commands/Magpjm/Commands/MagpjmCommands.php (partial)

<?php

namespace Drupal\magpjm\Commands;

use Drupal;
use Drupal\Core\StringTranslation\TranslatableMarkup;
use Drupal\magpjm\YearSetter;
use Drush\Commands\DrushCommands;

/**
 * MagpjmCommands provides Drush 9 commands for the MAGPJ run time module.
 */
class MagpjmCommands extends DrushCommands {

  use Drupal\Core\StringTranslation\StringTranslationTrait;

  /**
   * The magpjm.year_setter service.
   *
   * @var \Drupal\magpjm\YearSetter
   */
  protected $yearSetter;

  /**
   * MagpjmCommands constructor.
   *
   * @param \Drupal\magpjm\YearSetter $yearSetter
   *   The magpjm.year_setter service.
   */
  public function __construct(
    YearSetter $yearSetter,
  ) {
    $this->yearSetter = $yearSetter;
  }

  /**
   * Set the year on specific date fields on all entities in a bundle.
   *
   * @param string $entityType
   *   The type of the entities to update.
   * @param string $bundle
   *   The bundle of the entities to update.
   * @param string $fieldsString
   *   A comma-separated list of field machine names to update.
   * @param int $year
   *   The YYYY year to set on the specified fields.
   *
   * @validate-module-enabled magpjm
   *
   * @command magpjm:set-year
   * @aliases magpjm-set-year
   *
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   *   May happen on storage errors during batch updates.
   */
  public function setYear(string $entityType, string $bundle, string $fieldsString, int $year) {
    $fields = explode(',', $fieldsString);
    $this->yearSetter->update($entityType, $bundle, $fields, $year);
    drush_backend_batch_process();
  }

src/YearSetter.php

<?php

namespace Drupal\magpjm;

use Drupal\Component\Plugin\Exception\PluginNotFoundException;
use Drupal\Core\Entity\ContentEntityInterface;
use Drupal\Core\Entity\ContentEntityStorageInterface;
use Drupal\Core\Entity\EntityFieldManagerInterface;
use Drupal\Core\Entity\EntityTypeBundleInfoInterface;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Drupal\Core\Field\FieldDefinitionInterface;
use Drupal\Core\StringTranslation\StringTranslationTrait;
use Drupal\datetime\Plugin\Field\FieldType\DateTimeItemInterface;

/**
 * Controls a batch operation to update entities to a given year.
 *
 * It update specified date fields on all entities of a specified entity bundle.
 */
class YearSetter {

  use StringTranslationTrait;

  /**
   * Size of the entity array slice to process in one batch pass.
   */
  const SLICE = 50;

  /**
   * The entity_type.bundle.info service.
   *
   * @var \Drupal\Core\Entity\EntityTypeBundleInfoInterface
   */
  protected $bundleInfo;

  /**
   * The entity_field.manager service.
   *
   * @var \Drupal\Core\Entity\EntityFieldManagerInterface
   */
  protected $fieldManager;

  /**
   * The storage for the chosen entity type.
   *
   * @var \Drupal\Core\Entity\ContentEntityStorageInterface
   */
  protected $storage;

  /**
   * The entity_type.manager service.
   *
   * @var \Drupal\Core\Entity\EntityTypeManagerInterface
   */
  protected $entityTypeManager;

  /**
   * YearSetter constructor.
   *
   * @param \Drupal\Core\Entity\EntityTypeManagerInterface $entityTypeManager
   *   The entity_type.manager service.
   * @param \Drupal\Core\Entity\EntityTypeBundleInfoInterface $bundleInfo
   *   The entity_type.bundle_info service.
   * @param \Drupal\Core\Entity\EntityFieldManagerInterface $fieldManager
   *   The entity_field.manager service.
   */
  public function __construct(
    EntityTypeManagerInterface $entityTypeManager,
    EntityTypeBundleInfoInterface $bundleInfo,
    EntityFieldManagerInterface $fieldManager
  ) {
    $this->bundleInfo = $bundleInfo;
    $this->entityTypeManager = $entityTypeManager;
    $this->fieldManager = $fieldManager;
  }

  /**
   * Build the steps in the batch update for the year.
   *
   * @param string $bundle
   *   The entity bundle.
   * @param string $bundleKey
   *   The key naming the bundle.
   * @param array $fields
   *   The fields to update.
   * @param int $year
   *   The year to apply.
   *
   * @return array
   *   Batch steps.
   */
  protected function buildSteps(string $bundle, string $bundleKey, array $fields, int $year) {
    $ids = $this->storage->getQuery()
      ->condition($bundleKey, $bundle)
      ->execute();

    $stepBase = [
      [__CLASS__, 'updateChunk'],
      [$this->storage, $fields, $year],
    ];

    $idChunks = array_chunk($ids, static::SLICE);
    $steps = array_map(function (array $idChunk) use ($stepBase) {
      $step = $stepBase;
      $step[1][] = $idChunk;
      return $step;
    }, $idChunks);

    return $steps;
  }

  /**
   * Update fields on all entities in a given bundle with a new year.
   *
   * @param string $entityType
   *   The entity type id.
   * @param string $bundle
   *   The bundle id.
   * @param array $fields
   *   The fields to update.
   * @param int $year
   *   The year to apply.
   *
   * @return array
   *   A batch.
   *
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   *   May be thrown by the validateSelection() step on storage issues.
   * @throws \Drupal\Component\Plugin\Exception\PluginNotFoundException
   *   When the plugin cannot be loaded.
   */
  public function update(string $entityType, string $bundle, array $fields, int $year) {
    list($bundleKey, $this->storage) = $this->validateSelection($entityType, $bundle, $fields, $year);

    $operations = $this->buildSteps($bundle, $bundleKey, $fields, $year);

    $args = [
      '@entity' => $entityType,
      '@bundle' => $bundle,
      '@year' => $year,
    ];

    $batchInfo = ([
      'operations' => $operations,
      'title' => $this->t('Updating @entity/@bundle for @year', $args),
      'init_message' => $this->t('Initializing @year update.', $args),
      'progress_message' => $this->t('Update progress: @current/@total = @percentage% (remaining: @remaining)'),
      'error_message' => $this->t('An error occurred during the @year update.', $args),
      'progressive' => TRUE,
    ]);

    batch_set($batchInfo);
    $batch = &batch_get();
    return $batch;
  }

  /**
   * Batch step: update a chunk of entities.
   *
   * @param \Drupal\Core\Entity\ContentEntityStorageInterface $storage
   *   The entity storage for the entities to update.
   * @param array $fields
   *   The fields to update.
   * @param int $year
   *   The year to apply.
   * @param array $ids
   *   The ids of the entities in the chunk.
   * @param \DrushBatchContext|array $context
   *   The batch context.
   */
  public static function updateChunk(
    ContentEntityStorageInterface $storage,
    array $fields,
    int $year,
    array $ids,
    &$context) {
    $utime0 = microtime(TRUE);
    $entities = $storage->loadMultiple($ids);
    $formattedYear = sprintf('%04d', $year);
    array_walk($entities, function (ContentEntityInterface $entity) use ($fields, $formattedYear) {
      static::updateEntity($entity, $fields, $formattedYear);
    });

    $context['finished'] = TRUE;
    $msg = "Updating: " . implode(', ', $ids);
    $context['message'] = $msg;
    $context['results'][] = implode(', ', $ids);
    $utime1 = microtime(TRUE);

    // The batch system only refreshes once per second, so make updates
    // readable, but not too long.
    usleep(5E5 - $utime1 + $utime0);
  }

  /**
   * Update a single entity.
   *
   * @param \Drupal\Core\Entity\ContentEntityInterface $entity
   *   The entity to update.
   * @param array $fields
   *   The fields to update.
   * @param string $formattedYear
   *   The year to apply, already formatted.
   *
   * @throws \Drupal\Core\Entity\EntityStorageException
   *   In case saving encounters an error.
   */
  public static function updateEntity(ContentEntityInterface $entity, array $fields, string $formattedYear) {
    /** @var \Drupal\Core\Datetime\DrupalDateTime $field */
    foreach ($fields as $field) {
      /** @var \Drupal\Core\Field\FieldItemListInterface $f */
      $itemList = $entity->{$field};
      /** @var \Drupal\Core\Field\FieldItemInterface $item */
      foreach ($itemList as $item) {
        $oldValue = $item->value;
        $expectedValue = $formattedYear . substr($oldValue, 4);
        $newValue = (new \DateTime($expectedValue))->format(DateTimeItemInterface::DATE_STORAGE_FORMAT);
        if ($newValue !== $expectedValue) {
          // Do not update nodes to invalid dates.
          return;
        }
        $item->value = $newValue;
      }
    }
    $entity->save();
  }

  /**
   * Validate the settings to apply.
   *
   * @param string $entityType
   *   The entity type id.
   * @param string $bundle
   *   The bundle id.
   * @param array $fields
   *   The fields to update.
   * @param int $year
   *   The year to apply.
   *
   * @return array
   *   If checks have succeeded, return [bundle, entityStorage] for the selected
   *   entity type.
   *
   * @throws \AssertionError
   *   Thrown when some selections cannot be fulfilled.
   * @throws \Drupal\Component\Plugin\Exception\InvalidPluginDefinitionException
   *   When the storage handler cannot be used.
   * @throws \Drupal\Component\Plugin\Exception\PluginNotFoundException
   *   When the plugin cannot be loaded.
   */
  protected function validateSelection(string $entityType, string $bundle, array $fields, int $year) {
    assert($year >= 1970, "Year must be after the Epoch.");
    assert($year < 2038, "Year must be before 32-bit timestamp folding.");

    try {
      $definition = $this->entityTypeManager->getDefinition($entityType);
      $bundleKey = $definition->getKey('bundle');
    }
    catch (PluginNotFoundException $e) {
      throw new \AssertionError("Entity type \"${entityType}\" is not available.");
    }

    $bundles = $this->bundleInfo->getBundleInfo($entityType);
    assert(isset($bundles[$bundle]), "Bundle ${bundle} is not available on entity type ${entityType}.");

    $fieldsDefinitions = $this->fieldManager->getFieldDefinitions($entityType, $bundle);
    $missingFields = array_diff($fields, array_keys($fieldsDefinitions));
    assert(empty($missingFields), "Some requested fields are not available: " . implode(', ', $missingFields) . ".");

    $requestedFields = array_intersect_key($fieldsDefinitions, array_flip($fields));
    $nonDateFields = array_filter($requestedFields, function (FieldDefinitionInterface $definition) {
      return !in_array($definition->getType(), ['datetime', 'daterange']);
    });
    assert(empty($nonDateFields), "Some requested fields not not contain dates: " . implode(', ', array_keys($nonDateFields)) . ".");

    return [$bundleKey, $this->entityTypeManager->getStorage($entityType)];
  }

}

[mglaman]

Thanks!!! I'll try this.

Typically libraries which have functions like drush_backend_batch_process provide a functions.php to Composer for autoloading.

Like Guzzle:

    "autoload": {
        "psr-4": {
            "GuzzleHttp\\": "src/"
        },
        "files": [
            "src/functions_include.php"
        ]
    },

Does has nothing like that for https://github.com/drush-ops/drush/tree/master/includes :/

[mglaman]

😮 @fgm thanks for this. It made me realize the phpstan-drupal tests weren't running PHPStan rules mglaman/phpstan-drupal#63

So now I can add tests for this, properly.

[mglaman]

Working on the fix over in mglaman/phpstan-drupal#65, then I'll make a PR to take in that version for drupal-check and close this issue.

[fgm]

I think you mean "Drush" has nothing like that ? (not Does has nothing like that).

Also, I think this function problem is not related to the other issue with phpstan not finding the \DrushBatchContext class when checking the phpdoc for updateChunk, is it ?

[mglaman]

I think you mean "Drush" has nothing like that ? (not Does has nothing like that).

:) Phones

Also, I think this function problem is not related to the other issue with phpstan not finding the \DrushBatchContext class when checking the phpdoc for updateChunk, is it ?

Correct, that is PHPStan static analysis. By default drupal-check should only be running deprecations. If you run analysis that'll throw

[mglaman]

Fixed by #59