Package Porting

From OTRS 4 to 5
Kernel/Output/HTML restructured
Pre-Output-Filters
IE 8 and IE 9
GenericInterface API change in "Ticket" connector
Preview functions in dynamic statistics
HTML print discarded
Translation string extraction improved
From OTRS 3.3 to 4
New Object Handling
CacheInternalObject removed
Scheduler backend files moved
Update code sections in SOPM files
New Template Engine
New FontAwesome version
Unit Tests
Custom Ticket History types

With every new minor or major version of OTRS, you need to port your package(s) and make sure they still work with the OTRS API.

From OTRS 4 to 5

This section lists changes that you need to examine when porting your package from OTRS 4 to 5.

Kernel/Output/HTML restructured

In OTRS 5, Kernel/Output/HTML was restructured. All Perl modules (except Layout.pm) were moved to subdirectories (one for every module layer). Template (theme) files were also moved from Kernel/Output/HTML/Standard to Kernel/Output/HTML/Templates/Standard. Please perform this migration also in your code.

Pre-Output-Filters

With OTRS 5 there is no support for pre output filters any more. These filters changed the template content before it was parsed, and that could potentially lead to bad performance issues because the templates could not be cached any more and had to be parsed and compiled every time.

Just switch from pre to post output filters. To translate content, you can run $LayoutObject->Translate() directly. If you need other template features, just define a small template file for your output filter and use it to render your content before injecting it into the main data. It can also be helpful to use jQuery DOM operations to reorder/replace content on the screen in some cases instead of using regular expressions. In this case you would inject the new code somewhere in the page as invisible content (e. g. with the class Hidden), and then move it with jQuery to the correct location in the DOM and show it.

To make using post output filters easier, there is also a new mechanism to request HTML comment hooks for certain templates/blocks. You can add in your module config XML like:

<ConfigItem
Name="Frontend::Template::GenerateBlockHooks###100-OTRSBusiness-ContactWithData"
Required="1" Valid="1">
    <Description Translatable="1">Generate HTML comment hooks for
the specified blocks so that filters can use them.</Description>
    <Group>OTRSBusiness</Group>
    <SubGroup>Core</SubGroup>
    <Setting>
        <Hash>
            <Item Key="AgentTicketZoom">
                <Array>
                    <Item>CustomerTable</Item>
                </Array>
            </Item>
        </Hash>
    </Setting>
</ConfigItem>
                

This will cause the block CustomerTable in AgentTicketZoom.tt to be wrapped in HTML comments each time it is rendered:

<!--HookStartCustomerTable-->
... block output ...
<!--HookEndCustomerTable-->
                

With this mechanism every package can request just the block hooks it needs, and they are consistently rendered. These HTML comments can then be used in your output filter for easy regular expression matching.

IE 8 and IE 9

Support for IE 8 and 9 was dropped. You can remove any workarounds in your code for these platforms, as well as any old <CSS_IE7> or <CSS_IE8> loader tags that might still lurk in your XML config files.

GenericInterface API change in "Ticket" connector

The operation TicketGet() returns dynamic field data from ticket and articles differently than in OTRS 4. Now they are cleanly separated from the rest of the static ticket and article fields - they are now grouped in a list called DynamicField. Please adapt any applications using this operation accordingly.

# changed from:

Ticket => [
    {
        TicketNumber       => '20101027000001',
        Title              => 'some title',
        ...
        DynamicField_X     => 'value_x',
    },
]

# to:

Ticket => [
    {
        TicketNumber       => '20101027000001',
        Title              => 'some title',
        ...
        DynamicField => [
            {
                Name  => 'some name',
                Value => 'some value',
            },
        ],
    },
]
                

Preview functions in dynamic statistics

The new statistics GUI provides a preview for the current configuration. This must be implemented in the statistic modules and usually returns fake / random data for speed reasons. So for any dynamic (matrix) statistic that provides the method GetStatElement() you should also add a method GetStatElementPreview(), and for every dynamic (table) statistic that provides GetStatTable() you should accordingly add GetStatTablePreview(). Otherwise the preview in the new statistics GUI will not work for your statistics. You can find example implementations in the default OTRS statistics.

HTML print discarded

Until OTRS 5, the Perl module PDF::API2 was not present on all systems. Therefore a fallback HTML print mode existed. With OTRS 5, the module is now bundled and HTML print was dropped. $LayoutObject->PrintHeader() and PrintFooter() are not available any more. Please remove the HTML print fallback from your code and change it to generate PDF if necessary.

Translation string extraction improved

Until OTRS 5, translatable strings could not be extracted from Perl code and Database XML definitions. This is now possible and makes dummy templates like AAA*.tt obsolete. Please see this section for details.

From OTRS 3.3 to 4

This section lists changes that you need to examine when porting your package from OTRS 3.3 to 4.

New Object Handling

Up to OTRS 4, objects used to be created both centrally and also locally and then handed down to all objects by passing them to the constructors. With OTRS 4 and later versions, there is now an ObjectManager that centralizes singleton object creation and access.

This will require you first of all to change all top level Perl scripts (.pl files only!) to load and provide the ObjectManager to all OTRS objects. Let's look at otrs.CheckDB.pl from OTRS 3.3 as an example:

use strict;
use warnings;

use File::Basename;
use FindBin qw($RealBin);
use lib dirname($RealBin);
use lib dirname($RealBin) . '/Kernel/cpan-lib';
use lib dirname($RealBin) . '/Custom';

use Kernel::Config;
use Kernel::System::Encode;
use Kernel::System::Log;
use Kernel::System::Main;
use Kernel::System::DB;

# create common objects
my %CommonObject = ();
$CommonObject{ConfigObject} = Kernel::Config->new();
$CommonObject{EncodeObject} = Kernel::System::Encode->new(%CommonObject);
$CommonObject{LogObject}    = Kernel::System::Log->new(
    LogPrefix    => 'OTRS-otrs.CheckDB.pl',
    ConfigObject => $CommonObject{ConfigObject},
);
$CommonObject{MainObject} = Kernel::System::Main->new(%CommonObject);
$CommonObject{DBObject}   = Kernel::System::DB->new(%CommonObject);
                

We can see that a lot of code is used to load the packages and create the common objects that must be passed to OTRS objects to be used in the script. With OTRS 4, this looks quite different:

use strict;
use warnings;

use File::Basename;
use FindBin qw($RealBin);
use lib dirname($RealBin);
use lib dirname($RealBin) . '/Kernel/cpan-lib';
use lib dirname($RealBin) . '/Custom';

use Kernel::System::ObjectManager;

# create common objects
local $Kernel::OM = Kernel::System::ObjectManager->new(
    'Kernel::System::Log' => {
        LogPrefix => 'OTRS-otrs.CheckDB.pl',
    },
);

# get database object
my $DBObject = $Kernel::OM->Get('Kernel::System::DB');
                

The new code is a bit shorter than the old. It is no longer necessary to load all the packages, just the ObjectManager. Subsequently $Kernel::OM->Get('My::Perl::Package') can be used to get instances of objects which only have to be created once. The LogPrefix setting controls the log messages that Kernel::System::Log writes, it could also be omitted.

From this example you can also deduce the general porting guide when it comes to accessing objects: don't store them in $Self any more (unless needed for specific reasons). Just fetch and use the objects on demand like $Kernel::OM->Get('Kernel::System::Log')->Log(...). This also has the benefit that the Log object will only be created if something must be logged. Sometimes it could also be useful to create local variables if an object is used many times in a function, like $DBObject in the example above.

There's not much more to know when porting packages that should be loadable by the ObjectManager. They should declare the modules they use (via $Kernel::OM->Get()) like this:

our @ObjectDependencies = (
    'Kernel::Config',
    'Kernel::System::Log',
    'Kernel::System::Main',
);
                

The @ObjectDependencies declaration is needed for the ObjectManager to keep the correct order when destroying the objects.

Let's look at Valid.pm from OTRS 3.3 and 4 to see the difference. Old:

package Kernel::System::Valid;

use strict;
use warnings;

use Kernel::System::CacheInternal;

...

sub new {
    my ( $Type, %Param ) = @_;

    # allocate new hash for object
    my $Self = {};
    bless( $Self, $Type );

    # check needed objects
    for my $Object (qw(DBObject ConfigObject LogObject EncodeObject MainObject)) {
        $Self->{$Object} = $Param{$Object} || die "Got no $Object!";
    }

    $Self->{CacheInternalObject} = Kernel::System::CacheInternal->new(
        %{$Self},
        Type => 'Valid',
        TTL  => 60 * 60 * 24 * 20,
    );

    return $Self;
}

...

sub ValidList {
    my ( $Self, %Param ) = @_;

    # read cache
    my $CacheKey = 'ValidList';
    my $Cache = $Self->{CacheInternalObject}->Get( Key => $CacheKey );
    return %{$Cache} if $Cache;

    # get list from database
    return if !$Self->{DBObject}->Prepare( SQL => 'SELECT id, name FROM valid' );

    # fetch the result
    my %Data;
    while ( my @Row = $Self->{DBObject}->FetchrowArray() ) {
        $Data{ $Row[0] } = $Row[1];
    }

    # set cache
    $Self->{CacheInternalObject}->Set( Key => $CacheKey, Value => \%Data );

    return %Data;
}
                

New:

package Kernel::System::Valid;

use strict;
use warnings;

our @ObjectDependencies = (
    'Kernel::System::Cache',
    'Kernel::System::DB',
    'Kernel::System::Log',
);

...

sub new {
    my ( $Type, %Param ) = @_;

    # allocate new hash for object
    my $Self = {};
    bless( $Self, $Type );

    $Self->{CacheType} = 'Valid';
    $Self->{CacheTTL}  = 60 * 60 * 24 * 20;

    return $Self;
}

...

sub ValidList {
    my ( $Self, %Param ) = @_;

    # read cache
    my $CacheKey = 'ValidList';
    my $Cache    = $Kernel::OM->Get('Kernel::System::Cache')->Get(
        Type => $Self->{CacheType},
        Key  => $CacheKey,
    );
    return %{$Cache} if $Cache;

    # get database object
    my $DBObject = $Kernel::OM->Get('Kernel::System::DB');

    # get list from database
    return if !$DBObject->Prepare( SQL => 'SELECT id, name FROM valid' );

    # fetch the result
    my %Data;
    while ( my @Row = $DBObject->FetchrowArray() ) {
        $Data{ $Row[0] } = $Row[1];
    }

    # set cache
    $Kernel::OM->Get('Kernel::System::Cache')->Set(
        Type  => $Self->{CacheType},
        TTL   => $Self->{CacheTTL},
        Key   => $CacheKey,
        Value => \%Data
    );

    return %Data;
}
                

You can see that the dependencies are declared and the objects are only fetched on demand. We'll talk about the CacheInternalObject in the next section.

CacheInternalObject removed

Since Kernel::System::Cache is now also able to cache in-memory, Kernel::System::CacheInternal was dropped. Please see the previous example for how to migrate your code: you need to use the global Cache object and pass the Type settings with every call to Get(), Set(), Delete() and CleanUp(). The TTL parameter is now optional and defaults to 20 days, so you only have to specify it in Get() if you require a different TTL value.

Warning

It is especially important to add the Type to CleanUp() as otherwise not just the current cache type but the entire cache would be deleted.

Scheduler backend files moved

The backend files of the scheduler moved from Kernel/Scheduler to Kernel/System/Scheduler. If you have any custom Task Handler modules, you need to move them also.

Update code sections in SOPM files

Code tags in SOPM files have to be updated. Please do not use $Self any more. In the past this was used to get access to OTRS objects like the MainObject. Please use the ObjectManager now. Here is an example for the old style:

<CodeInstall Type="post">

# define function name
my $FunctionName = 'CodeInstall';

# create the package name
my $CodeModule = 'var::packagesetup::' . $Param{Structure}->{Name}->{Content};

# load the module
if ( $Self->{MainObject}->Require($CodeModule) ) {

    # create new instance
    my $CodeObject = $CodeModule->new( %{$Self} );

    if ($CodeObject) {

        # start method
        if ( !$CodeObject->$FunctionName(%{$Self}) ) {
            $Self->{LogObject}->Log(
                Priority => 'error',
                Message  => "Could not call method $FunctionName() on $CodeModule.pm."
            );
        }
    }

    # error handling
    else {
        $Self->{LogObject}->Log(
            Priority => 'error',
            Message  => "Could not call method new() on $CodeModule.pm."
        );
    }
}

</CodeInstall>
                

Now this should be replaced by:

<CodeInstall Type="post"><![CDATA[
$Kernel::OM->Get('var::packagesetup::MyPackage')->CodeInstall();
]]></CodeInstall>
                

New Template Engine

With OTRS 4, the DTL template engine was replaced by Template::Toolkit. Please refer to the Templating section for details on how the new template syntax looks like.

These are the changes that you need to apply when converting existing DTL templates to the new Template::Toolkit syntax:

Table 4.1. Template Changes from OTRS 3.3 to 4

DTL Tag Template::Toolkit tag
$Data{"Name"} [% Data.Name %]
$Data{"Complex-Name"} [% Data.item("Complex-Name") %]
$QData{"Name"} [% Data.Name | html %]
$QData{"Name", "$Length"} [% Data.Name | truncate($Length) | html %]
$LQData{"Name"} [% Data.Name | uri %]
$Quote{"Text", "$Length"} cannot be replaced directly, see examples below
$Quote{"$Config{"Name"}"} [% Config("Name") | html %]
$Quote{"$Data{"Name"}", "$Length"} [% Data.Name | truncate($Length) | html %]
$Quote{"$Data{"Content"}","$QData{"MaxLength"}"} [% Data.Name | truncate(Data.MaxLength) | html %]
$Quote{"$Text{"$Data{"Content"}"}","$QData{"MaxLength"}"} [% Data.Content | Translate | truncate(Data.MaxLength) | html %]
$Config{"Name"} [% Config("Name") %]
$Env{"Name"} [% Env("Name") %]
$QEnv{"Name"} [% Env("Name") | html %]
$Text{"Text with %s placeholders", "String"} [% Translate("Text with %s placeholders", "String") | html %]
$Text{"Text with dynamic %s placeholders", "$QData{Name}"} [% Translate("Text with dynamic %s placeholders", Data.Name) | html %]
'$JSText{"Text with dynamic %s placeholders", "$QData{Name}"}' [% Translate("Text with dynamic %s placeholders", Data.Name) | JSON %]
"$JSText{"Text with dynamic %s placeholders", "$QData{Name}"}" [% Translate("Text with dynamic %s placeholders", Data.Name) | JSON %]
$TimeLong{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("TimeLong") %]
$TimeShort{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("TimeShort") %]
$Date{"$Data{"CreateTime"}"} [% Data.CreateTime | Localize("Date") %]
<-- dtl:block:Name -->...<-- dtl:block:Name --> [% RenderBlockStart("Name") %]...[% RenderBlockEnd("Name") %]
<-- dtl:js_on_document_complete -->...<-- dtl:js_on_document_complete --> [% WRAPPER JSOnDocumentComplete %]...[% END %]
<-- dtl:js_on_document_complete_placeholder --> [% PROCESS JSOnDocumentCompleteInsert %]
$Include{"Copyright"} [% InsertTemplate("Copyright") %]

There is also a helper script bin/otrs.MigrateDTLtoTT.pl that will automatically port the DTL files to Template::Toolkit syntax for you. It might fail if you have errors in your DTL, please correct these first and re-run the script afterwards.

There are a few more things to note when porting your code to the new template engine:

  • All language files must now have the use utf8; pragma.

  • Layout::Get() is now deprecated. Please use Layout::Translate() instead.

  • All occurrences of $Text{""} in Perl code must now be replaced by calls to Layout::Translate().

    This is because in DTL there was no separation between template and data. If DTL-Tags were inserted as part of some data, the engine would still parse them. This is no longer the case in Template::Toolkit, there is a strict separation of template and data.

    Hint: should you ever need to interpolate tags in data, you can use the Interpolate filter for this ([% Data.Name | Interpolate %]). This is not recommended for security and performance reasons!

  • For the same reason, dynamically injected JavaScript that was enclosed by dtl:js_on_document_complete will not work any more. Please use Layout::AddJSOnDocumentComplete() instead of injecting this as template data.

    You can find an example for this in Kernel/System/DynamicField/Driver/BaseSelect.pm.

  • Please be careful with pre output filters (the ones configured in Frontend::Output::FilterElementPre). They still work, but they will prevent the template from being cached. This could lead to serious performance issues. You should definitely not have any pre output filters that operate on all templates, but limit them to certain templates via configuration setting.

    The post output filters (Frontend::Output::FilterElementPost) don't have such strong negative performance effects. However, they should also be used carefully, and not for all templates.

New FontAwesome version

With OTRS 4, we've also updated FontAwesome to a new version. As a consequence, the icons CSS classes have changed. While previously icons were defined by using a schema like icon-{iconname}, it is now fa fa-{iconname}.

Due to this change, you need to make sure to update all custom frontend module registrations which make use of icons (e.g. for the top navigation bar) to use the new schema. This is also true for templates where you're using icon elements like <i class="icon-{iconname}"></i>.

Unit Tests

With OTRS 4, in Unit Tests $Self no longer provides common objects like the MainObject, for example. Please always use $Kernel::OM->Get('...') to fetch these objects.

Custom Ticket History types

If you use any custom ticket history types, you have to take two steps for them to be displayed correctly in AgentTicketHistory of OTRS 4+.

Firstly, you have to register your custom ticket history types via SysConfig. This could look like:

<ConfigItem Name="Ticket::Frontend::HistoryTypes###100-MyCustomModule" Required="1" Valid="1">
    <Description Translatable="1">Controls how to display the ticket history entries as readable values.</Description>
    <Group>Ticket</Group>
    <SubGroup>Frontend::Agent::Ticket::ViewHistory</SubGroup>
    <Setting>
        <Hash>
            <Item Key="MyCustomType" Translatable="1">Added information (%s)</Item>
        </Hash>
    </Setting>
</ConfigItem>
                

The second step is to translate the English text that you provided for the custom ticket history type in your translation files, if needed. That's it!

If you are interested in the details, please refer to this commit for additional information about the changes that happened in OTRS.