Package Porting

From OTRS 3.3 to 4
New Object Handling
CacheInternalObject removed
Scheduler backend files moved
Update code sections in SOPM files
New Template Engine
New FontAwesome version
UnitTests
Custom TicketHistory types

With every new minor or major version of OTRS, you need to port your package(s) and make sure they still work with the OTRS API.

From OTRS 3.3 to 4

This section lists changes that you need to examine when porting your package from OTRS 3.3 to 4.

New Object Handling

Up to OTRS 4, objects used to be created both centrally and also locally and then handed down to all objects by passing them to the contructors. With OTRS 4 and later versions, there is now an ObjectManager that centralizes singleton object creation and access.

This will require you first of all to change all toplevel Perl scripts (.pl files only!) to load and provide the ObjectManager to all OTRS objects. Let's look at otrs.CheckDB.pl from OTRS 3.3 as an example:

use strict;
use warnings;

use File::Basename;
use FindBin qw($RealBin);
use lib dirname($RealBin);
use lib dirname($RealBin) . '/Kernel/cpan-lib';
use lib dirname($RealBin) . '/Custom';

use Kernel::Config;
use Kernel::System::Encode;
use Kernel::System::Log;
use Kernel::System::Main;
use Kernel::System::DB;

# create common objects
my %CommonObject = ();
$CommonObject{ConfigObject} = Kernel::Config->new();
$CommonObject{EncodeObject} = Kernel::System::Encode->new(%CommonObject);
$CommonObject{LogObject}    = Kernel::System::Log->new(
    LogPrefix    => 'OTRS-otrs.CheckDB.pl',
    ConfigObject => $CommonObject{ConfigObject},
);
$CommonObject{MainObject} = Kernel::System::Main->new(%CommonObject);
$CommonObject{DBObject}   = Kernel::System::DB->new(%CommonObject);
                

We can see that a lot of code is used to load the packages and create the common objects that must be passed to OTRS objects to be used in the script. With OTRS 4, this looks quite different:

use strict;
use warnings;

use File::Basename;
use FindBin qw($RealBin);
use lib dirname($RealBin);
use lib dirname($RealBin) . '/Kernel/cpan-lib';
use lib dirname($RealBin) . '/Custom';

use Kernel::System::ObjectManager;

# create common objects
local $Kernel::OM = Kernel::System::ObjectManager->new(
    'Kernel::System::Log' => {
        LogPrefix => 'OTRS-otrs.CheckDB.pl',
    },
);

# get database object
my $DBObject = $Kernel::OM->Get('Kernel::System::DB');
                

The new code is a bit shorter than the old. It is no longer necessary to load all the packages, just the ObjectManager. Subsequently $Kernel::OM->Get('My::Perl::Package') can be used to get instances of Objects which only have to be created once. The LogPrefix setting controls the log messages that Kernel::System::Log writes, it could also be omitted.

From this example you can also deduce the general porting guide when it comes to accessing objects: don't store them in $Self any more (unless needed for specific reasons). Just fetch and use the objects on demand like $Kernel::OM->Get('Kernel::System::Log')->Log(...). This also has the benefit that the Log object will only be created if something must be logged. Sometimes it could also be useful to create local variables if an object is used many times in a function, like $DBObject in the example above.

There's not much more to know when porting packages that should be loadable by the ObjectManager. They should declare the modules they use (via $Kernel::OM->Get()) like this:

our @ObjectDependencies = (
    'Kernel::Config',
    'Kernel::System::Log',
    'Kernel::System::Main',
);
                

The @ObjectDependencies declaration is needed for the ObjectManager to keep the correct order when destroying the objects.

Let's look at Valid.pm from OTRS 3.3 and 4 to see the difference. Old:

package Kernel::System::Valid;

use strict;
use warnings;

use Kernel::System::CacheInternal;

...

sub new {
    my ( $Type, %Param ) = @_;

    # allocate new hash for object
    my $Self = {};
    bless( $Self, $Type );

    # check needed objects
    for my $Object (qw(DBObject ConfigObject LogObject EncodeObject MainObject)) {
        $Self->{$Object} = $Param{$Object} || die "Got no $Object!";
    }

    $Self->{CacheInternalObject} = Kernel::System::CacheInternal->new(
        %{$Self},
        Type => 'Valid',
        TTL  => 60 * 60 * 24 * 20,
    );

    return $Self;
}

...

sub ValidList {
    my ( $Self, %Param ) = @_;

    # read cache
    my $CacheKey = 'ValidList';
    my $Cache = $Self->{CacheInternalObject}->Get( Key => $CacheKey );
    return %{$Cache} if $Cache;

    # get list from database
    return if !$Self->{DBObject}->Prepare( SQL => 'SELECT id, name FROM valid' );

    # fetch the result
    my %Data;
    while ( my @Row = $Self->{DBObject}->FetchrowArray() ) {
        $Data{ $Row[0] } = $Row[1];
    }

    # set cache
    $Self->{CacheInternalObject}->Set( Key => $CacheKey, Value => \%Data );

    return %Data;
}
                

New:

package Kernel::System::Valid;

use strict;
use warnings;

our @ObjectDependencies = (
    'Kernel::System::Cache',
    'Kernel::System::DB',
    'Kernel::System::Log',
);

...

sub new {
    my ( $Type, %Param ) = @_;

    # allocate new hash for object
    my $Self = {};
    bless( $Self, $Type );

    $Self->{CacheType} = 'Valid';
    $Self->{CacheTTL}  = 60 * 60 * 24 * 20;

    return $Self;
}

...

sub ValidList {
    my ( $Self, %Param ) = @_;

    # read cache
    my $CacheKey = 'ValidList';
    my $Cache    = $Kernel::OM->Get('Kernel::System::Cache')->Get(
        Type => $Self->{CacheType},
        Key  => $CacheKey,
    );
    return %{$Cache} if $Cache;

    # get database object
    my $DBObject = $Kernel::OM->Get('Kernel::System::DB');

    # get list from database
    return if !$DBObject->Prepare( SQL => 'SELECT id, name FROM valid' );

    # fetch the result
    my %Data;
    while ( my @Row = $DBObject->FetchrowArray() ) {
        $Data{ $Row[0] } = $Row[1];
    }

    # set cache
    $Kernel::OM->Get('Kernel::System::Cache')->Set(
        Type  => $Self->{CacheType},
        TTL   => $Self->{CacheTTL},
        Key   => $CacheKey,
        Value => \%Data
    );

    return %Data;
}
                

You can see that the dependencies are declared and the objects are only fetched on demand. We'll talk about the CacheInternalObject in the next section.

CacheInternalObject removed

Since Kernel::System::Cache is now also able to cache in-memory, Kernel::System::CacheInternal was dropped. Please see the previous example for how to migrate your code: you need to use the global Cache object and pass the Type settings with every call to Get(), Set(), Delete() and CleanUp(). The TTL parameter is now optional and defaults to 20 days, so you only have to specify it in Get() if you require a different TTL value.

Warning

It is especially important to add the Type to CleanUp() as otherwise not just the current cache type but the entire cache would be deleted.

Scheduler backend files moved

The backend files of the scheduler moved from Kernel/Scheduler to Kernel/System/Scheduler. If you have any custom TaskHandler modules, you need to move them also.

Update code sections in SOPM files

Code tags in SOPM files have to be updated. Please do not use $Self any more. In the past this was used to get access to OTRS objects like the MainObject. Please use the ObjectManager now. Here is an example for the old style:

<CodeInstall Type="post">

# define function name
my $FunctionName = 'CodeInstall';

# create the package name
my $CodeModule = 'var::packagesetup::' . $Param{Structure}->{Name}->{Content};

# load the module
if ( $Self->{MainObject}->Require($CodeModule) ) {

    # create new instance
    my $CodeObject = $CodeModule->new( %{$Self} );

    if ($CodeObject) {

        # start method
        if ( !$CodeObject->$FunctionName(%{$Self}) ) {
            $Self->{LogObject}->Log(
                Priority => 'error',
                Message  => "Could not call method $FunctionName() on $CodeModule.pm."
            );
        }
    }

    # error handling
    else {
        $Self->{LogObject}->Log(
            Priority => 'error',
            Message  => "Could not call method new() on $CodeModule.pm."
        );
    }
}

</CodeInstall>
                

Now this should be replaced by:

<CodeInstall Type="post"><![CDATA[
$Kernel::OM->Get('var::packagesetup::MyPackage')->CodeInstall();
]]></CodeInstall>
                

New Template Engine

With OTRS 4, the DTL template engine was replaced by Template::Toolkit. Please refer to the Templating section for details on how the new template syntax looks like.

These are the changes that you need to apply when converting existing DTL templates to the new Template::Toolkit syntax:

Table 4.1. Template Changes from OTRS 3.3 to 4

DTL Tag Template::Toolkit tag
$Data{"Name"} [% Data.Name %]
$Data{"Complex-Name"} [% Data.item("Complex-Name") %]
$QData{"Name"} [% Data.Name | html %]
$QData{"Name", "$Length"} [% Data.Name | truncate($Length) | html %]
$LQData{"Name"} [% Data.Name | uri %]
$Quote{"Text", "$Length"} cannot be replaced directly, see examples below
$Quote{"$Config{"Name"}"} [% Config("Name") | html %]
$Quote{"$Data{"Name"}", "$Length"} [% Data.Name | truncate($Length) | html %]
$Quote{"$Data{"Content"}","$QData{"MaxLength"}"} [% Data.Name | truncate(Data.MaxLength) | html %]
$Quote{"$Text{"$Data{"Content"}"}","$QData{"MaxLength"}"} [% Data.Content | Translate | truncate(Data.MaxLength) | html %]
$Config{"Name"} [% Config("Name") %]
$Env{"Name"} [% Env("Name") %]
$QEnv{"Name"} [% Env("Name") | html %]
$Text{"Text with %s placeholders", "String"} [% Translate("Text with %s placeholders", "String") | html %]
$Text{"Text with dynamic %s placeholders", "$QData{Name}"} [% Translate("Text with dynamic %s placeholders", Data.Name) | html %]
'$JSText{"Text with dynamic %s placeholders", "$QData{Name}"}' [% Translate("Text with dynamic %s placeholders", Data.Name) | JSON %]
"$JSText{"Text with dynamic %s placeholders", "$QData{Name}"}" [% Translate("Text with dynamic %s placeholders", Data.Name) | JSON %]
$TimeLong{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("TimeLong") %]
$TimeShort{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("TimeShort") %]
$Date{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("Date") %]
<-- dtl:block:Name -->...<-- dtl:block:Name --> [% RenderBlockStart("Name") %]...[% RenderBlockEnd("Name") %]
<-- dtl:js_on_document_complete -->...<-- dtl:js_on_document_complete --> [% WRAPPER JSOnDocumentComplete %]...[% END %]
<-- dtl:js_on_document_complete_placeholder --> [% PROCESS JSOnDocumentCompleteInsert %]
$Include{"Copyright"} [% InsertTemplate("Copyright") %]

There is also a helper script bin/otrs.MigrateDTLtoTT.pl that will automatically port the DTL files to Template::Toolkit syntax for you. It might fail if you have errors in your DTL, please correct these first and re-run the script afterwards.

There are a few more things to note when porting your code to the new template engine:

  • All language files must now have the use utf8; pragma.

  • Layout::Get() is now deprecated. Please use Layout::Translate() instead.

  • All occurrences of $Text{""} in Perl code must now be replaced by calls to Layout::Translate().

    This is becase in DTL there was no separation between template and data. If DTL-Tags were inserted as part of some data, the engine would still parse them. This is no longer the case in Template::Toolkit, there is a strict separation of template and data.

    Hint: should you ever need to interpolate tags in data, you can use the Interpolate filter for this ([% Data.Name | Interpolate %]). This is not recommended for security and performance reasons!

  • For the same reason, dynamically injected JavaScript that was enclosed by dtl:js_on_document_complete will not work any more. Please use Layout::AddJSOnDocumentComplete() instead of injecting this as template data.

    You can find an example for this in Kernel/System/DynamicField/Driver/BaseSelect.pm.

  • Please be careful with "pre" output filters (the ones configured in Frontend::Output::FilterElementPre). They still work, but they will prevent the template from being cached. This could lead to serious performance issues. You should definitely not have any pre output filters that operate on all templates, but limit them to certain templates via configuration setting.

    Post output filters (Frontend::Output::FilterElementPost don't have such strong negative performance effects. However, they should also be used carefully, and not for all templates.

New FontAwesome version

With OTRS 4, we've also updated FontAwesome to a new version. As a consequence, the icons css classes have changed. While previously icons were defined by using a schema like icon-{iconname}, it is now fa fa-{iconname}.

Due to this change, you need to make sure to update all custom frontend module registrations which make use of icons (e.g. for the top navigation bar) to use the new schema. This is also true for templates where you're using icon elements like <i class="icon-{iconname}"></i>.

UnitTests

With OTRS 4, in UnitTests $Self no longer provides common objects like the MainObject, for example. Please always use $Kernel::OM->Get('...') to fetch these objects.

Custom TicketHistory types

If you use any custom ticket history types, you have to take two steps for them to be displayed correctly in AgentTicketHistory of OTRS 4+.

Firstly, you have to register your custom ticket history types via SysConfig. This could look like:

<ConfigItem Name="Ticket::Frontend::HistoryTypes###100-MyCustomModule" Required="1" Valid="1">
    <Description Translatable="1">Controls how to display the ticket history entries as readable values.</Description>
    <Group>Ticket</Group>
    <SubGroup>Frontend::Agent::Ticket::ViewHistory</SubGroup>
    <Setting>
        <Hash>
            <Item Key="MyCustomType" Translatable="1">Added information (%s)</Item>
        </Hash>
    </Setting>
</ConfigItem>
                

The second step is to translate the english text that you provided for the custom ticket history type in your translation files, if needed. That's it!

If you are interested in the details, please refer to this commit for additional information about the changes that happened in OTRS.