Release 2.4.5-p1-13.9

TAG: R2.4.5-p1-13.9

branch: main

Release date: February 2024

This release upgrades version R2.4.5-p1-12.5

Backorders and procurement value

Introduction

Orders that have the status processing can be viewed as backorders. The order was allowed and registered but has yet to be shipped. Since, in most orders, customers pay upfront, we may have a disconnect between the revenue and cost of revenue (procurement value).

It is paramount to know how many orders are still in backorder status and what the total (approximated) procurement value is so that we have an eye on our cash flow development.

Database queries

Simple database queries learn which orders are backorders and what their total value entails.

SELECT SUM(`base_grand_total`) 
FROM `sales_order` 
WHERE`status` = 'processing';

SELECT `entity_id`, `increment_id`, `base_grand_total`
FROM `sales_order`
WHERE `status` ='processing';

With these queries, we can access all information in the sales_order table to create a new table, backorders_order, to contain the data extracted from the sales order. Once the table is available, the total amount for all orders is found as follows:

SELECT SUM(`base_grand_total`) FROM `backorders_order`;

Updating the backorders table

Initially, we implemented a ‘brute force’ scheme to update the table. It simply drops all table entries and creates a new table. Although this may seem a clever solution, it has a severe disadvantage: the incremental IDs continue to be added. There is no reason to do this, and not desired either.

Therefore, we have rewritten the scheme to be more elegant:

    private function updateBackorderWorker(): bool
    {
        $status = Constants::SUCCESS;
        $orders = $this->backorderOperations->getOrdersQualifyingForBackorder();
        $backorders = $this->backorderOperations->getBackorders();

        $backordersToDelete = $this->backorderOperations->getBackordersForDeletion($backorders);
        $ordersToAddAsBackorder = $this->backorderOperations->getOrdersForAddition($orders, $backorders);
        $backordersToSave = $this->backorderOperations->convertOrdersToBackorders($ordersToAddAsBackorder);

        $status = $status && $this->backorderOperations->deleteBackorders($backordersToDelete);
        return $status && $this->backorderOperations->saveBackorders($backordersToSave);
    }

It seems more work, but it is robust and clean. We gather all backorders as orders from the sales order table that qualify as a backorder: status='processing'. Then, collect all current backorders from the backorder table.

With these two lists, we can find out which backorders no longer qualify - we add them to the list to delete:

    /**
     * Find backorders to delete
     *
     * @param array $backorders
     * @return array
     */
    public function getBackordersForDeletion(array $backorders): array
    {
        $backordersToDelete = [];
        foreach ($backorders as $backorder) {
            $order_id = $backorder['order_id'];
            try {
                $order = $this->orderRepository->get($order_id);
                if ($order->getStatus() != 'processing') {
                    $backordersToDelete [] = $backorder;
                }
            } catch (InputException|NoSuchEntityException) {
                $backordersToDelete[] = $backorder;
            }
        }

        return $backordersToDelete;
    }

Then, we find out which qualifying orders from the sales order table are not yet available as a backorder: we add them to the list of orders as a backorder:

    /**
     * Find orders to add as a backorder
     *
     * @param array $qualifyingOrders
     * @param array $backorders
     * @return array
     */
    public function getOrdersForAddition(array $qualifyingOrders, array $backorders): array
    {
        $ordersToAdd = [];
        $orderIds = $this->getOrderIdsFromBackorders($backorders);
        foreach ($qualifyingOrders as $order) {
            $order_id = $order->getEntityId();
            $key = in_array($order_id, $orderIds);
            if (!$key) {
                $ordersToAdd[] = $order;
            }
        }

        return $ordersToAdd;
    }

You have to convert the orders to add to backorders; after conversion, we can save them. Finally, we cleaned up the table by deleting the backorders that are no longer backorders.

You can also use the same scheme as the action we connect to the Update button on the backorder overview page in the backend.

Below we detail how we exactly implement all parts.

Declarative schema

Since Magento 2.3, the database schema’s maintenance is run through the “declarative schema.” This makes maintenance more manageable as the system will create the required mySQL queries to bring the database to the required state. See the Magento Development Documentation for more detailed info.

You can find the details here. You find examples of using the declarative schema here.

Create a new module: Cloudscraper/Backorders

We use the online tool: https://mage2gen.com:

It is easy enough to create a new module, in this case, with the name Backorders.

Creation of the new database table

When you create a model with the Mage2Gen generator, it creates all code necessary. What it does is the following:

The name is used to create the database table (modulename_modelname).
It automatically creates a primary entity id (modelname_id).
The field you define is added to the table.
Every field you define with the same model name adds the field to the table.

In the interface, this looks like this:

The tool creates several files with code, one of which is etc/db_schema.xml:

<?xml version="1.0" ?>
<schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Setup/Declaration/Schema/etc/schema.xsd">
    <table name="cloudscraper_backorders_backorder" resource="default" engine="innodb" comment="cloudscraper_backorders_backorder Table">
        <column xsi:type="int" name="backorder_id" padding="10" unsigned="true" nullable="false" identity="true" comment="Entity Id"/>
        <constraint xsi:type="primary" referenceId="PRIMARY">
            <column name="backorder_id"/>
        </constraint>
        <column name="increment_id" nullable="true" xsi:type="varchar" comment="Increment ID" length="255"/>
        <column name="base_grand_total" nullable="true" xsi:type="decimal" comment="Base Grand Total" scale="4" precision="12"/>
        <column name="order_id" nullable="true" xsi:type="int" comment="Order ID" identity="false"/>
        <column name="base_subtotal" nullable="true" xsi:type="decimal" comment="base_subtotal" scale="4" precision="12"/>
        <column name="updated_at" nullable="true" xsi:type="datetime" comment="updated_at"/>
    </table>
</schema>

Based on this, you can create the schema whitelist as outlined below.

When you run the upgrade command, the system creates the table accordingly. With the information to create the table, and the fact we checked the adminhtml grid option, we also get the code to create an entry in the menu for, in our case Cloudscraper, with the option Backorders, and a grid is created showing the content.

The table name is: cloudscraper_backorders_backorder. With this information, the system can create the database table once you use the bin/magento setup:upgrade command.

Before we can run the Magento upgrade command, we need to create a schema whitelist for our table. This ensures the table can be adequately maintained (see here for more information):

The whitelist sequence

bin/magento module:enable Cloudscraper_Backorders
bin/magento cache:clean
bin/magento setup:db-declaration:generate-whitelist --module-name="Cloudscraper_Backorders"

Then, run the Magento2 upgrade command:

bin/magento setup:upgrade

After this command finishes, the new table is created. Should you need to add more fields, add them to the model with the same name (in this case, Backorder), and the field information is added to the db_schema.xml file. After these changes, update the schema whitelist:

bin/magento setup:db-declaration:generate-whitelist --module-name="Cloudscraper_Backorders"

We can create test data by inserting values into the backorders table as:

INSERT INTO `cloudscraper_backorders_backorder` 
   (`increment_id`,`base_grand_total`,`order_id`)
    SELECT `increment_id`, `base_grand_total`,`entity_id` as order_id
    FROM `sales_order` WHERE `status` = 'processing';

The model and the repository

When you create the model (to define the table) in Mage2Gen, the tool simultaneously creates the information required to code the model and the repository. It results in the interface files found at Cloudscraper\Backorders\Api

The interfaces define how we communicate with the database. It contains the following files:

Api\BackorderRepositoryInterface

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Api;

use Magento\Framework\Api\SearchCriteriaInterface;

interface BackorderRepositoryInterface
{

    /**
     * Save Backorder
     * @param \Cloudscraper\Backorders\Api\Data\BackorderInterface $backorder
     * @return \Cloudscraper\Backorders\Api\Data\BackorderInterface
     * @throws \Magento\Framework\Exception\LocalizedException
     */
    public function save(
        \Cloudscraper\Backorders\Api\Data\BackorderInterface $backorder
    );

    /**
     * Retrieve Backorder
     * @param string $backorderId
     * @return \Cloudscraper\Backorders\Api\Data\BackorderInterface
     * @throws \Magento\Framework\Exception\LocalizedException
     */
    public function get($backorderId);

    /**
     * Retrieve Backorder matching the specified criteria.
     * @param \Magento\Framework\Api\SearchCriteriaInterface $searchCriteria
     * @return \Cloudscraper\Backorders\Api\Data\BackorderSearchResultsInterface
     * @throws \Magento\Framework\Exception\LocalizedException
     */
    public function getList(
        \Magento\Framework\Api\SearchCriteriaInterface $searchCriteria
    );

    /**
     * Delete Backorder
     * @param \Cloudscraper\Backorders\Api\Data\BackorderInterface $backorder
     * @return bool 
     * @throws \Magento\Framework\Exception\LocalizedException
     */
    public function delete(
        \Cloudscraper\Backorders\Api\Data\BackorderInterface $backorder
    );

    /**
     * Delete Backorder by ID
     * @param string $backorderId
     * @return bool
     * @throws \Magento\Framework\Exception\NoSuchEntityException
     * @throws \Magento\Framework\Exception\LocalizedException
     */
    public function deleteById($backorderId);
}

You find the basic CRUD options on a backorder object from the above.

Api/Data/BackorderInterface.php

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Api\Data;

interface BackorderInterface
{

    const ORDER_ID = 'order_id';
    const BACKORDER_ID = 'backorder_id';
    const UPDATED_AT = 'updated_at';
    const INCREMENT_ID = 'increment_id';
    const BASE_SUBTOTAL = 'base_subtotal';
    const BASE_GRAND_TOTAL = 'base_grand_total';

    /**
     * Get backorder_id
     * @return string|null
     */
    public function getBackorderId();

    /**
     * Set backorder_id
     * @param string $backorderId
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setBackorderId($backorderId);

    /**
     * Get increment_id
     * @return string|null
     */
    public function getIncrementId();

    /**
     * Set increment_id
     * @param string $incrementId
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setIncrementId($incrementId);

    /**
     * Get base_grand_total
     * @return string|null
     */
    public function getBaseGrandTotal();

    /**
     * Set base_grand_total
     * @param string $baseGrandTotal
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setBaseGrandTotal($baseGrandTotal);

    /**
     * Get order_id
     * @return string|null
     */
    public function getOrderId();

    /**
     * Set order_id
     * @param string $orderId
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setOrderId($orderId);

    /**
     * Get base_subtotal
     * @return string|null
     */
    public function getBaseSubtotal();

    /**
     * Set base_subtotal
     * @param string $baseSubtotal
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setBaseSubtotal($baseSubtotal);

    /**
     * Get updated_at
     * @return string|null
     */
    public function getUpdatedAt();

    /**
     * Set updated_at
     * @param string $updatedAt
     * @return \Cloudscraper\Backorders\Backorder\Api\Data\BackorderInterface
     */
    public function setUpdatedAt($updatedAt);
}

The backorderInterface class defines the backorder and the getter and setter methods. You can create a backorder object like in the code below:

    /**
     * Convert order to backorder object worker
     *
     * @param OrderInterface $order
     * @return BackorderInterface
     */
    private function convertOrderToBackOrder(OrderInterface $order): BackorderInterface
    {
        $backorderModel = $this->backorderFactory->create();
        $this->setBackorderData($backorderModel, $order);
        return $backorderModel;
    }

The last interface created with the model is:

Api/Data/BackorderSearchResultsInterface.php

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Api\Data;

interface BackorderSearchResultsInterface extends \Magento\Framework\Api\SearchResultsInterface
{

    /**
     * Get Backorder list.
     * @return \Cloudscraper\Backorders\Api\Data\BackorderInterface[]
     */
    public function getItems();

    /**
     * Set increment_id list.
     * @param \Cloudscraper\Backorders\Api\Data\BackorderInterface[] $items
     * @return $this
     */
    public function setItems(array $items);
}

With this interface, you have access to the collection. From the interface, you can see how this works; the result is an array of BackorderInterface objects.

The way you use this is as follows:

    /**
     * Get backorders from the sales_order table
     *
     * @return array
     * @throws LocalizedException
     */
    public function getBackorders(): array
    {
        $searchCriteria = $this->getSearchCriteriaBackorderValueNotNull();
        $backorders = $this->backorderRepository->getList($searchCriteria)->getItems();
        return $this->checkBackordersForRelatedOrder($backorders);
    }

Using the repository and the search criteria, you can get a list of items, and the getItems() method creates the array with the backorder objects.

You can set the search criteria as follows:

    /**
     * Set the criteria to get backorders
     *
     * @return SearchCriteria
     */
    private function getSearchCriteriaBackorderValueNotNull(): SearchCriteria
    {
        $filter = $this->filterBuilder
            ->setField(BackorderInterface::BASE_GRAND_TOTAL)
            ->setConditionType('notnull')
            ->create();
        $this->searchCriteriaBuilder->addFilters([$filter]);
        return $this->searchCriteriaBuilder->create();
    }

Another example of setting the search criteria:

    /**
     * Define the status=processing search criterion
     *
     * @return SearchCriteria
     */
    private function getSearchCriteriaOrderIsProcessing(): SearchCriteria
    {
        $filter = $this->filterBuilder
            ->setField(OrderInterface::STATUS)
            ->setConditionType('eq')
            ->setValue('processing')
            ->create();
        $this->searchCriteriaBuilder->addFilters([$filter]);
        return $this->searchCriteriaBuilder->create();
    }

Then, when you feed the SearchCriteria object to the getList() method, you get the list of items from the database that satisfy your search criteria. Use the getItems() method to materialize the list.

Note: Interfaces do not define code, but just the interface. The Model directory thus contains files that, indeed implement the interfaces as described above.

Presentation in the Magento backend - the grid

You can use a controller to define the view to get a grid in the backend. The overview page will be connected to the index controller action in this case. We need a block where we can keep the data to create the appropriate view. You can define it in Mage2Gen as a block like this:

The code that comes with it: Block/Adminhtml/Backorders.php

And the layout is defined here: view/adminhtml/layout/cloudscraper_backorders_backorder_index.xml We have added the generated layout file to fit our needs; the module provides the basics. Here is what we finally use:

<?xml version="1.0" ?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <body>
        <referenceContainer name="content">
            <block class="Cloudscraper\Backorders\Block\Adminhtml\Backorders"
                   name="backorders" as="backorders"
                   template="Cloudscraper_Backorders::backorders.phtml" >
                <block name="backorder.shared.summary"
                       class="Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared\Summary"
                       template="Cloudscraper_Backorders::backorder/shared/summary.phtml" />
            </block>
            <uiComponent name="cloudscraper_backorders_backorder_listing"/>
        </referenceContainer>
    </body>
    <update handle="styles"/>
</page>

You notice two blocks in the content reference container, backorders, and backorder.shared.summary. The latter we have added, because it contains code that we (initially) used in two controller actions. You also see it defines template files that are associated with the block classes (Cloudscraper\Backorders\Block\Adminhtml\Backorders and Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared\Summary). These classes contain the code you need to present the summary data of the list of backorders.

Note: The summary.phtml template is a child of the backorders.phtml as reflected by the getChildHtml() method in the parent template (see code below).

Adding the uiComponent tag that refers to cloudscraper_backorders_backorder_listing creates the grid. The code for this ui-component is found in view/adminhtml/ui_component/cloudscraper_backorders_backorder_listing.xml and it defines how the grid should look like:

<?xml version="1.0" ?>
<listing xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <argument name="data" xsi:type="array">
        <item name="js_config" xsi:type="array">
            <item name="provider" xsi:type="string">
                cloudscraper_backorders_backorder_listing.cloudscraper_backorders_backorder_listing_data_source</item>
        </item>
        <item name="buttons" xsi:type="array">
            <item name="add" xsi:type="array">
                <item name="name" xsi:type="string">update</item>
                <item name="label" xsi:type="string" translate="true">Update</item>
                <item name="class" xsi:type="string">primary</item>
                <item name="url" xsi:type="string">*/*/update</item>
            </item>
        </item>
    </argument>
    <settings>
        <spinner>cloudscraper_backorders_backorder_columns</spinner>
        <deps>
            <dep>cloudscraper_backorders_backorder_listing.cloudscraper_backorders_backorder_listing_data_source</dep>
        </deps>
    </settings>
    <dataSource name="cloudscraper_backorders_backorder_listing_data_source" component="Magento_Ui/js/grid/provider">
        <settings>
            <storageConfig>
                <param name="indexField" xsi:type="string">backorder_id</param>
            </storageConfig>
            <updateUrl path="mui/index/render"/>
        </settings>
        <aclResource>Cloudscraper_Backorders::Backorder</aclResource>
        <dataProvider name="cloudscraper_backorders_backorder_listing_data_source"
                      class="Magento\Framework\View\Element\UiComponent\DataProvider\DataProvider">
            <settings>
                <requestFieldName>id</requestFieldName>
                <primaryFieldName>backorder_id</primaryFieldName>
            </settings>
        </dataProvider>
    </dataSource>
    <listingToolbar name="listing_top">
        <settings>
            <sticky>true</sticky>
        </settings>
        <bookmark name="bookmarks"/>
        <columnsControls name="columns_controls"/>
        <filters name="listing_filters"/>
        <paging name="listing_paging"/>
    </listingToolbar>
    <columns name="cloudscraper_backorders_backorder_columns">
        <selectionsColumn name="ids">
            <settings>
                <indexField>backorder_id</indexField>
            </settings>
        </selectionsColumn>
        <column name="backorder_id">
            <settings>
                <filter>text</filter>
                <sorting>asc</sorting>
                <label translate="true">ID</label>
            </settings>
        </column>
        <column name="increment_id">
            <settings>
                <filter>text</filter>
                <label translate="true">Order</label>
            </settings>
        </column>
        <column name="base_grand_total">
            <settings>
                <filter>text</filter>
                <label translate="true">Backorder value (incl. TAX)</label>
            </settings>
        </column>
        <column name="base_subtotal">
            <settings>
                <filter>text</filter>
                <label translate="true">Nett backorder value (excl. TAX)</label>
            </settings>
        </column>
        <column name="updated_at">
            <settings>
                <filter>text</filter>
                <label translate="true">Order last updated</label>
            </settings>
        </column>
        <column name="order_id">
            <settings>
                <filter>text</filter>
                <label translate="true">Order ID</label>
                <visible>false</visible>
            </settings>
        </column>
        <actionsColumn name="actions" class="Cloudscraper\Backorders\Ui\Component\Listing\Grid\Column\Action">
            <argument name="data" xsi:type="array">
                <item name="config" xsi:type="array">
                    <item name="resizeEnabled" xsi:type="boolean">true</item>
                    <item name="resizeDefaultWidth" xsi:type="string">120</item>
                    <item name="indexField" xsi:type="string">id</item>
                </item>
            </argument>
        </actionsColumn>
    </columns>
</listing>

The Update button

When you inspect the above code, you will find that we have added parts not generated by the Mage2Gen code generator. First, the Update button with code like this:

<item name="buttons" xsi:type="array">
    <item name="add" xsi:type="array">
        <item name="name" xsi:type="string">update</item>
        <item name="label" xsi:type="string" translate="true">Update</item>
        <item name="class" xsi:type="string">primary</item>
        <item name="url" xsi:type="string">*/*/update</item>
    </item>
</item>

The url is defined with two wildcards, which follow from the route_id defined in the routes.xml file. The front name is cloudscraper_backorders. Thus making the first part of the url: cloudscraper_backorders. The second part is the section name (or controller name), which is backorder in our case. The action name update completes the url: cloudscraper_backorders/backorder/update and is placed on the button.

From the Magento documentation:

<store-url>/<store-code>/<front-name>/<controller-name>/<action-name>

This is defined in etc/adminhtml/routes.xml:

<?xml version="1.0" ?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:App/etc/routes.xsd">
    <router id="admin">
        <route id="cloudscraper_backorders" frontName="cloudscraper_backorders">
            <module name="Cloudscraper_Backorders" before="Magento_Backend"/>
        </route>
    </router>
</config>

Thus, once the button is pushed, the controller action is called, and you will run the execute() method in the Update.php file located at Controller/Adminhtml/Backorder/Update.php.

The index controller is a simple redirect to the update controller:

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Controller\Adminhtml\Backorder;

use Magento\Backend\App\Action;
use Magento\Framework\Controller\Result\Redirect;

class Index extends Action
{
    /**
     * Redirect to the updated backorder overview
     *
     * @return Redirect
     */
    public function execute()
    {
        $resultRedirect = $this->resultRedirectFactory->create();
        $resultRedirect->setPath('cloudscraper_backorders/backorder/update');
        return $resultRedirect;
    }
}

The content of the Update class:

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Controller\Adminhtml\Backorder;

use Cloudscraper\Backorders\Block\Adminhtml\Backorder\Update as UpdateBlock;
use Magento\Backend\App\Action;
use Magento\Backend\App\Action\Context;
use Magento\Framework\Exception\LocalizedException;
use Magento\Framework\View\Result\Page;
use Magento\Framework\View\Result\PageFactory;

class Update extends Action
{

    /**
     * @var PageFactory
     */
    protected PageFactory $resultPageFactory;
    /**
     * @var UpdateBlock
     */
    private UpdateBlock $updateBlock;

    /**
     * Constructor
     *
     * @param UpdateBlock $updateBlock
     * @param Context $context
     * @param PageFactory $resultPageFactory
     */
    public function __construct(
        UpdateBlock      $updateBlock,
        Context     $context,
        PageFactory $resultPageFactory
    ) {
        $this->resultPageFactory = $resultPageFactory;
        parent::__construct($context);
        $this->updateBlock = $updateBlock;
    }

    /**
     * Update & show backorder overview
     *
     * @return Page
     * @throws LocalizedException
     */
    public function execute(): Page
    {
        $this->updateBlock->updateBackorders();
        $resultPage = $this->resultPageFactory->create();
        $resultPage->getConfig()->getTitle()->prepend(__("Backorder overview"));
        return $resultPage;
    }
}

You can read from the above code how it works: when the index controller is called (cloudscraper_backorders/backorder/update), it first updates the backorder table before presenting them in the grid. This way, the overview (index) and the update action ensure you always see the up-to-date information. The code to update the table is injected by dependency injection, and concerns the Update block class (in the above code called updateBlock):

<?php
/**
 * Copyright © Cloudscraper Software, Arthur CJ Venis, M.Sc. All rights reserved.
 * See COPYING.txt for license details.
 */
declare(strict_types=1);

namespace Cloudscraper\Backorders\Block\Adminhtml\Backorder;

use Cloudscraper\Backorders\Model\BackorderOperations;
use Cloudscraper\Core\Constants\Constants;
use Magento\Backend\Block\Template;
use Magento\Backend\Block\Template\Context;
use Magento\Framework\Exception\LocalizedException;

class Update extends Template
{
    /**
     * @var BackorderOperations
     */
    private BackorderOperations $backorderOperations;

    /**
     * Constructor
     *
     * @param BackorderOperations $backorderOperations
     * @param Context $context
     * @param array $data
     */
    public function __construct(
        BackorderOperations $backorderOperations,
        Context             $context,
        array               $data = []
    ) {
        parent::__construct($context, $data);
        $this->backorderOperations = $backorderOperations;
    }

    /**
     * Update backorder table
     *
     * @return bool
     * @throws LocalizedException
     */
    public function updateBackorders(): bool
    {
        $status = Constants::SUCCESS;
        return $status && $this->updateBackorderWorker();
    }

    /**
     * Update backorder table worker
     *
     * @return bool
     * @throws LocalizedException
     */
    private function updateBackorderWorker(): bool
    {
        $status = Constants::SUCCESS;
        $orders = $this->backorderOperations->getOrdersQualifyingForBackorder();
        $backorders = $this->backorderOperations->getBackorders();

        $backordersToDelete = $this->backorderOperations->getBackordersForDeletion($backorders);
        $ordersToAddAsBackorder = $this->backorderOperations->getOrdersForAddition($orders, $backorders);
        $backordersToSave = $this->backorderOperations->convertOrdersToBackorders($ordersToAddAsBackorder);

        $status = $status && $this->backorderOperations->deleteBackorders($backordersToDelete);
        return $status && $this->backorderOperations->saveBackorders($backordersToSave);
    }
}

In the above you recognize the code fragment we started this explanation with. The actual work is done by the worker that depends on the BackorderOperations model class. This class ‘knows’ how to get to the data of the backorders and persist them when required.

When done, the ResultsPage that the controller returns is taken by the UI-component, creating the generic Magento grid, with our information.

Template files

We find more information on the backorders in the grid representation (see the figure above). This information comes from the template files we have added and defined in the view/adminhtml/layout/cloudscraper_backorders_backorder_update.xml file. It concerns view/adminhtml/templates/backorders.phtml and view/adminhtml/templates/backorder/shared/summary.phtml.

Note: the layout files follow a naming convention: view/adminhtml/layout/cloudscraper_backorders_backorder_update.xml belongs to the update controller with url: cloudscraper_backorders/backorder/update.

The content of these files:

<?php
/**
 * @var $block Backorders
 */

use Cloudscraper\Backorders\Block\Adminhtml\Backorders;
echo $block->getChildHtml();

and

<?php
/*
 * Copyright (c) 2023.
 * Arthur CJ Venis M.Sc.
 * Cloudscraper Software
 */

/**
 * @var $block Summary
 */
use Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared\Summary;
use Cloudscraper\Backorders\Helper\Constants as BackorderConstants;
use Cloudscraper\Core\Constants\Constants;

/** @var array $totalBackordersValue */
$totalBackordersValue = $block->getBackordersTotalValue();
?>

<div>
    <span><b><?= $escaper->escapeHtml(__('Total sales value of backorders (incl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::TOTAL_VALUE_INCL_TAX]); ?>
</div>
<div>
    <br/>
</div>
<div>
    <span><b><?= $escaper->escapeHtml(__('Estimated cashflow impact (incl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::ESTIMATED_INCL_TAX]); ?>
</div>
<div>
    <span><b><?= $escaper->escapeHtml(__('Estimated procurement value (excl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::ESTIMATED_EXCL_TAX]); ?>
</div>

The first file does not do much since we have delegated everything to the shared summary template. The latter uses the Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared\Summary class for the $block and makes the needed functionality available to the HTML:

<?php
/*
 * Copyright (c) 2023.
 * Arthur CJ Venis M.Sc.
 * Cloudscraper Software
 */

namespace Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared;

use Cloudscraper\Backorders\Api\Data\BackorderInterface;
use Cloudscraper\Backorders\Helper\Constants as BackorderConstants;
use Cloudscraper\Backorders\Model\BackorderOperations;
use Cloudscraper\Core\Constants\Constants;
use Cloudscraper\Core\Helper\PhpFunctionWrapper;
use Magento\Backend\Block\Template;
use Magento\Backend\Block\Template\Context;
use Magento\Framework\Exception\LocalizedException;

class Summary extends Template
{
    /**
     * @var PhpFunctionWrapper
     */
    private PhpFunctionWrapper $phpFunctionWrapper;
    /**
     * @var BackorderOperations
     */
    private BackorderOperations $backorderOperations;

    /**
     * @param BackorderOperations $backorderOperations
     * @param PhpFunctionWrapper $phpFunctionWrapper
     * @param Context $context
     * @param array $data
     */
    public function __construct(
        BackorderOperations $backorderOperations,
        PhpFunctionWrapper  $phpFunctionWrapper,
        Context             $context,
        array               $data = []
    ) {
        parent::__construct($context, $data);
        $this->phpFunctionWrapper = $phpFunctionWrapper;
        $this->backorderOperations = $backorderOperations;
    }

    /**
     * Get backorder total sales value
     *
     * @return array
     * @throws LocalizedException
     */
    public function getBackordersTotalValue(): array
    {
        $backorders = $this->backorderOperations->getBackorders();
        $totalBackordersValue = [
            BackorderConstants::TOTAL_VALUE_INCL_TAX => Constants::ZERO,
            BackorderConstants::TOTAL_VALUE_EXCL_TAX => Constants::ZERO,
            BackorderConstants::ESTIMATED_INCL_TAX => Constants::ZERO,
            BackorderConstants::ESTIMATED_EXCL_TAX => Constants::ZERO
        ];
        foreach ($backorders as $backorder) {
            $totalBackordersValue[BackorderConstants::TOTAL_VALUE_INCL_TAX] +=
                $backorder[BackorderInterface::BASE_GRAND_TOTAL];
            $totalBackordersValue[BackorderConstants::TOTAL_VALUE_EXCL_TAX] +=
                $backorder[BackorderInterface::BASE_SUBTOTAL];
        }

        $totalBackordersValue[BackorderConstants::ESTIMATED_EXCL_TAX] =
            $this->getBackordersEstimatedProcurementValue($totalBackordersValue);
        $totalBackordersValue[BackorderConstants::ESTIMATED_INCL_TAX] =
            $this->getValueInclTax($totalBackordersValue[BackorderConstants::ESTIMATED_EXCL_TAX]);
        return $this->roundOffCurrencyValue($totalBackordersValue);
    }

    /**
     * Get backorder estimated procurement value
     *
     * @param array $totalBackordersValue
     * @return float|int
     */
    private function getBackordersEstimatedProcurementValue(array $totalBackordersValue): float|int
    {
        return $totalBackordersValue[BackorderConstants::TOTAL_VALUE_EXCL_TAX] *
            BackorderConstants::PROCUREMENT_PERCENTAGE;
    }

    /**
     * Get a value including tax
     *
     * @param float|int $value
     * @return float|int
     */
    private function getValueInclTax(float|int $value): float|int
    {
        return $value*BackorderConstants::TAX_21;
    }

    /**
     * Round off a currency value to two decimals
     *
     * @param array $array
     * @return array
     */
    private function roundOffCurrencyValue(array $array): array
    {
        $returnArray = [];
        foreach ($array as $key => $value) {
            $returnArray[$key] = $this->phpFunctionWrapper->numberFormat($value, 2);
        }
        return $returnArray;
    }
}

Action column in the grid

When you look at the grid, you can see the last column contains a link with a connected action. This action column is defined in the layout view/adminhtml/ui_component/cloudscraper_backorders_backorder_listing.xml.

    <actionsColumn name="actions" class="Cloudscraper\Backorders\Ui\Component\Listing\Grid\Column\Action">
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="resizeEnabled" xsi:type="boolean">true</item>
                <item name="resizeDefaultWidth" xsi:type="string">120</item>
                <item name="indexField" xsi:type="string">id</item>
            </item>
        </argument>
    </actionsColumn>

In the columns tag, the actionColumn defines where the action is defined when the link is used: Cloudscraper\Backorders\Ui\Component\Listing\Grid\Column\Action class contains the code that defines the action:

<?php
/*
 * Copyright (c) 2023.
 * Arthur CJ Venis M.Sc.
 * Cloudscraper Software
 */

namespace Cloudscraper\Backorders\Ui\Component\Listing\Grid\Column;

use Magento\Framework\UrlInterface;
use Magento\Framework\View\Element\UiComponent\ContextInterface;
use Magento\Framework\View\Element\UiComponentFactory;
use Magento\Ui\Component\Listing\Columns\Column;

class Action extends Column
{
    /**
     * Url path
     *
     */
    public const SALES_ORDER_VIEW_ORDER_ID = 'sales/order/view/order_id';

    /** @var UrlInterface */
    protected UrlInterface $_urlBuilder;

    /**
     * @var string
     */
    private string $_editUrl;

    /**
     * @param ContextInterface $context
     * @param UiComponentFactory $uiComponentFactory
     * @param UrlInterface $urlBuilder
     * @param array $components
     * @param array $data
     * @param string $editUrl
     */
    public function __construct(
        ContextInterface   $context,
        UiComponentFactory $uiComponentFactory,
        UrlInterface       $urlBuilder,
        array              $components = [],
        array              $data = [],
        string             $editUrl = self::SALES_ORDER_VIEW_ORDER_ID
    ) {
        $this->_urlBuilder = $urlBuilder;
        $this->_editUrl = $editUrl;
        parent::__construct($context, $uiComponentFactory, $components, $data);
    }

    /**
     * Prepare the action column view order url link in the grid
     *
     * @param array $dataSource
     * @return array
     */
    public function prepareDataSource(array $dataSource): array
    {
        if (isset($dataSource['data']['items'])) {
            $viewUrlPath = $this->_editUrl;
            $urlEntityParamName = 'order_id';
            $itemName = $this->getData('name');
            foreach ($dataSource['data']['items'] as & $item) {
                if (isset($item['order_id'])) {
                    $item[$itemName] = [
                        'view' => [
                            'href' => $this->_urlBuilder->getUrl(
                                $viewUrlPath,
                                [
                                    $urlEntityParamName => $item['order_id']
                                ]
                            ),
                            'label' => __('Viewer'),
                        ]
                    ];
                }
            }
        }

        return $dataSource;
    }
}

The prepareDataSource() method sets up the link in the last part of the code. We want to view the original order that is also a backorder, when clicking the view url. The url for doing so: sales/order/view/order_id. This requires the entity_id of the original order, which we stored as order_id in the backorder table. The code constructs the data source for all entries in the grid. The UI-component then includes the correct url in the grid.

We have created the backend menu in a fashion that we can reuse. In the Core module, we have added the menu.xml file in the etc/adminhtml directory:

<?xml version="1.0" ?>
<!--
  ~ Copyright (c) 2023.
  ~ Arthur CJ Venis M.Sc.
  ~ Cloudscraper Software
  -->

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Backend:etc/menu.xsd">
    <menu>
        <add id="Cloudscraper::top_level" title="avelanche" 
             module="Cloudscraper_Core" sortOrder="50" resource="Magento_Backend::content"/>

        <add id="Cloudscraper_Core::new_action" title="Other module" translate="title" 
             module="Cloudscraper_Core" parent="Cloudscraper::top_level" sortOrder="20" 
             dependsOnModule="Cloudscraper_Core" resource="Magento_Backend::content"/>
            <add id="Cloudscraper_Core::cloudscraper_core_update" title="Action 1" translate="title" 
                 module="Cloudscraper_Core" resource="Magento_Backend::content" parent="Cloudscraper_Core::new_action"/>
            <add id="Cloudscraper_Core::cloudscraper_core_clear" title="Action 2" translate="title" 
                 module="Cloudscraper_Core" resource="Magento_Backend::content" parent="Cloudscraper_Core::new_action"/>

    </menu>
</config>

This creates the backend menu with the title avelanche and two dummy items underneath a dummy submenu for now.

In the Backorders module, you can find the etc/adminhtml/menu.xml file with the following content:

<?xml version="1.0" ?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Backend:etc/menu.xsd">
    <menu>
        <add id="Cloudscraper_Backorders::backorders"
             title="Backorders" translate="title" 
             module="Cloudscraper_Backorders" parent="Cloudscraper::top_level" sortOrder="10" 
             dependsOnModule="Cloudscraper_Backorders" resource="Magento_Backend::content"/>
            <add id="Cloudscraper_Backorders::cloudscraper_backorders_backorder" title="Overview" translate="title" 
                 module="Cloudscraper_Backorders" resource="Magento_Backend::content" 
                 parent="Cloudscraper_Backorders::backorders" 
                 action="cloudscraper_backorders/backorder/index"/>
     </menu>
</config>

In the above file, the first item is placed under the parent Cloudscraper::top_level, defined in the menu.xml of the core module. This way, we place the access to the Backorders module in the backend under the generic avelanche backend menu. The second item Cloudscraper_Backorders::cloudscraper_backorders_backorder defines an action url: cloudscraper_backorders/backorder/index. In the earlier explanation about the controller we have seen what happens when this url is called.

Backorder total value & estimated procurement value

All backorders’ total sales value (incl. VAT) is the sum of the base_grand_total of all open backorders. The number includes everything in the order, also the shipping cost the customer pays. The estimated procurement value is based on the sum of the nett sales value of all orders (i.e., the value of the orders excluding VAT and shipping cost, the base_subtotal), and the average margin percentage (39.5%). The impact on the cashflow is the total procurement value, including VAT.

These numbers are computed and then presented in the template files. More specifically, in the shared/summary.phtml file:

<?php
/*
 * Copyright (c) 2023.
 * Arthur CJ Venis M.Sc.
 * Cloudscraper Software
 */

/**
 * @var $block Summary
 */
use Cloudscraper\Backorders\Block\Adminhtml\Backorder\Shared\Summary;
use Cloudscraper\Backorders\Helper\Constants as BackorderConstants;
use Cloudscraper\Core\Constants\Constants;

/** @var array $totalBackordersValue */
$totalBackordersValue = $block->getBackordersTotalValue();
?>

<div>
    <span><b><?= $escaper->escapeHtml(__('Total sales value of backorders (incl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::TOTAL_VALUE_INCL_TAX]); ?>
</div>
<div>
    <br/>
</div>
<div>
    <span><b><?= $escaper->escapeHtml(__('Estimated cashflow impact (incl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::ESTIMATED_INCL_TAX]); ?>
</div>
<div>
    <span><b><?= $escaper->escapeHtml(__('Estimated procurement value (excl. TAX):')); ?> </b></span>
    <?= $escaper->escapeHtml(Constants::EURO_SIGN . $totalBackordersValue[BackorderConstants::ESTIMATED_EXCL_TAX]); ?>
</div>

As we explained above, the layout file couples the template and the block, in this case the summary.phtml template and the Summary.php class contains the logic to calculate the numbers we present in the overview.

Note: We use the $escaper variable in the above code. Since Magento 2.4 this variable is available in every template file. This way, you can easily use the escaper functions to avoid XSS.

Languages

By default, you should include the two languages we have on our site: English (United States), and Dutch (Netherlands). This results in a directory i18n in the module’s root. You will have two files in this directory when you create the two language sets: en_US.csv and nl_NL.csv.

The content of these files can be defined from your code like follows:

bin/magento i18n:collect-phrases app/code/Cloudscraper/Core/ -o app/code/Cloudscraper/Core/i18n/en_US.csv
bin/magento i18n:collect-phrases app/code/Cloudscraper/Core/ -o app/code/Cloudscraper/Core/i18n/nl_NL.csv

The above command collects all instances of calls to the translation method (__('string to translate')) and the locations where we request translation in the xml files, like this:

    <column name="base_grand_total">
        <settings>
            <filter>text</filter>
            <label translate="true">Backorder value (incl. TAX)</label>
        </settings>
    </column>

The content of the translation files will then look like this (e.g., the nl_NL.csv file):

"Backorder overview","Overzicht nabestellingen"
"Could not save the backorder: %1","Kan backorder %1 niet opslaan."
"Backorder with id ""%1"" does not exist.","Backorder met id ""%1"" bestaat niet."
"Could not delete the Backorder: %1","Kan backorder %1 niet verwijderen."
"Viewer",Bekijk
"Total sales value of backorders (incl. TAX):","Totale verkoopwaarde (incl. btw):"
"Estimated cashflow impact (incl. TAX):","Geschatte cashflow impact (incl. btw):"
"Estimated procurement value (excl. TAX):","Geschatte inkoopwaarde (excl. btw):"
Update,Update
ID,ID
Order,Order
"Backorder value (incl. TAX)","Verkoopwaarde (incl. btw)"
"Nett backorder value (excl. TAX)","Netto waarde (excl. btw)"
"Order last updated","Order laatst aangepast"
"Order ID","Order ID"

Webkul Magento 2 Code Generator

For this project, we tried to use the Webkul Magento2 Code Generator.

We can install the code generator as require-dev in the composer setup:

composer2 require webkul/code-generator --no-update --no-dev

By doing so, we have the option to leave out the code generator in the production code. We need to update the deployment script to do so:

composer2 install --no-dev

With the above command, you install everything needed except what is denoted as required for development only.

NOTE: we currently already do so. We must fix the development deployment as this also leaves out the development-required code.

Updates to the code generator

We found some issues with the code generator. For example, when you try to create the code for the model, the generator crashes with an error on a null argument that is not allowed in the laminas code.

We can fix this by creating our code generator, and use wrapper functions to catch the null arguments.

NOTE: Unfortunately this does not work; the functions that pass the null argument are static. The solution is to create an extension class for the classes that have the methods that pass the argument and change the argument to an empty string instead.

Furthermore, the comment blocks in the generated code contain hard-wired information on the original developer. This is correct for the code generator itself, but you want your own copyright and author names for the generated code. We can add our own dist files to correct this.

Using Mage2Gen code generator

We found the Webkul Code Generator has some flaws, so we looked for an alternative. We found the code generator. It’s an online generator that creates code live while you add the items you need.

You can find the code generator here.

This is the tool we have used to further develop the basic scaffolding of the new module. Webkul is no longer used for this purpose. We removed the module from the system.

Option label removed from the pdf-files

The files we send with mail (order, invoice, shipment, and credit memo) should all have the same layout when it comes to the presentation of the products ordered, invoiced, shipped, or credited. They are now all the same, and similar to the order pdf-file.

MyParcel carrier module

We use MyParcel more frequently to send our products to customers. The track & trace code for MyParcel lands on a special art’s excellence page. We have added a module to the backend that makes entering the trace code easy. Just the trace code suffices and the system will create the correct link to the landing page that holds the detailed track & trace information on the shipment.

Cloudscraper Software - Magento 2 development

How to use and develop the Cloudscraper system