Zammad Package Best Practices

Development
1

There seems to be lots of movement around packages the past months! This is really great for the community.

Here are a few I know of:

I’m in the process of refactoring our Signal and WhatsApp channels from our zammad repo into packages, so I hope to add them to that list soon!

We’ve learned some things about how best to structure packages, and how to achieve certain things. However there’s a few questions I have on how best to refactor certain changes from the source tree into a package:

Unknown

These are things we still don’t know how to do. @thorsteneckel perhaps you can shed some light on these?

1. How to update config.active_record.observers in config/application.rb?

We want to add the signal and whatsapp observers into config/application.rb. How do we do this with a package?

see: config/application.rb · 78704ed91975984abd9a155a6dec2868edb2501d · Center for Digital Resilience / Link / zammad · GitLab

2. How to add icons?

All the SVG icons are combined into one file at public/assets/images/icons.svg. How can we add an icon to this file, or otherwise make it available (without overwriting the entire file)?

You can see here how we add a custom article type action for replying with Signal, using the signal icon. This works when we can add the icon to public/assets/images/icons.svg. Is there another search path for icons?

3. Testing

Once you move your changes out of the source tree, it becomes difficult to test of course.

We’ve solved this by including tests inside the szpm, installing the package on a dev machine and running rake test like normal.

Is there a better way?

Also, it would be SUPER amazing if you all could share your CI base image (registry.znuny.com/docker/zammad-ruby:2.5.5) or at least the Dockerfile for it. Then we could setup a CI pipeline for packages that includes the selenium tests and everything else.

How to…

Here is what we do know! Hopefully this helps someone.

Migrations

As per thorsten’s info:

You need to place them in db/addon/your_package_name and need to define self.up and/or self.down on them. Rails change method will not work!
For testing purposes you can use Package::Migration.migrate('YourPackageName') or Package::Migration.migrate('YourPackageName', 'reverse') on the Zammad rails console.

Here is an example migration.

You can achieve a lot with migrations, for example:

  • Set default system settings
  • Create new Object Attributes
  • Anything you can do from the rails console

CSS

CSS files should go in app/assets/stylesheets/addons/. SCSS is also supported.

General file overwrite

You can overwrite any file in the installation, however if you overwrite official files you will not receive updates when the Zammad Core team changes them. So generally speaking it is best not to overwrite existing files.

Auto installation

If you place your .zpm file into RAILS_ROOT/auto_install, then it will be automatically installed when the rake db:seed task is executed. You can trigger the auto install your self from the console Package.auto_install.

Note: Until PR #2933 is merged, the extension of the package must be .zpm.

5 Likes
2

Hi @abelxluck - thanks a ton for this comprehensive writeup of the current state. Feels really good :slight_smile: I’ll answer inline:

1. How to update config.active_record.observers in config/application.rb ?

We haven’t had this use case yet. I guess the proper way would be to use a Rails initializer and hooks to manipulate either the config or the target variable. You can see how they are doing it here. However, there are two thinks I want to share: a) we’re planning to migrate away from observers because they have been removed from Rails core and are just regular callbacks under the hood anyway. Additionally we will migrate to Rails::Engine based addons to allow enable to full power of integration they bring. Both changes have no final date and have not been started yet. Contributions are very welcome. Happy to share further information if required or requested.

2. How to add icons?

Same here: We haven’t had the use case yet. We’re will overhaul the whole assets and their processing in the near future. However, this will take some time. We’re open to ideas and contributions on how to improve the current situation in the meantime.

3. Testing

Adding tests (in the form op RSpec tests!) is the right way to go. We have dedicated repositories for the addons of our customers with custom CI images that will install the package and run the CI. We will share those images but need some time to get things in shape and migrate our processes accordingly.

How to…

Yeah! That’s basically how we do it. One thing regarding Auto installation: We link packages into the Zammad folder via Package.link. This way we can keep our package repositories and the main Framework repository separate. However, it comes with one caveat: You need to restart your processes after you changed files because the OS does not trigger FS events for links of which the original file has changed. We’re currently working around this by applying the following diff when developing packages:

diff --git a/config/environments/development.rb b/config/environments/development.rb
index 0167a307e..835b68832 100644
--- a/config/environments/development.rb
+++ b/config/environments/development.rb
@@ -54,7 +54,8 @@ Rails.application.configure do
 
   # Use an evented file watcher to asynchronously detect changes in source code,
   # routes, locales, etc. This feature depends on the listen gem.
-  config.file_watcher = ActiveSupport::EventedFileUpdateChecker
+  # config.file_watcher = ActiveSupport::EventedFileUpdateChecker
+  config.file_watcher = ActiveSupport::FileUpdateChecker
 
   # Asset digests allow you to set far-future HTTP expiration dates on all assets,
   # yet still be able to expire them through the digest params.

We’re working on a more sustainable solution but this will take some time. As always: Contributions welcome!

The ZAMMAD_ROOT/auto_install path is meant for deployment but not development. Does our described approach solve your issue? I tend not to merge your PR TBH.

Bests,
Thorsten

1 Like
3

Answers inline :slight_smile:

1. How to update config.active_record.observers in config/application.rb ?

To be honest I am not a rails developer and am floudering around when it comes to adding features like these. I only added the observer for our Signal and WhatsApp channels because that’s how the telegram channel did it (I used that channel as a base for the others). But if I understand its purpose, it is necessary to ensure the background task runs to fetch new messages (our channels use a polling mechanism, not a webhook mechanism like Telegram).

We definitely need these now, we rely on these channels daily, so while a future move away from observers is fine, we need a solution today. We really hate maintaining our own fork of Zammad, we hope to be able to do everything with packages.

My proposed solution is: Since config.active_record.observers is just an array of strings, we can easily add items to that array at runtime. Perhaps a directory config/observers/ that contains files containing new line separated observers that should be appended to the list. How does that sound? Should we move this to a GH issue?

2. How to add icons?

I think a similar solution to what I proposed above could be used here. Since public/assets/images/icons.svg is a bunch of <symbol /> elements wrapped in a <svg> tag. We could create the dir public/assets/images/icons-svg/, put text files containing the appropriate svg snippet inside, and then at build time concat those into icons.svg appending them to the end of the already existing ones. Should I make a GH ticket + PR?

3. Testing

Really looking forward to that! Our lack of automated testing on the addons is really slowing us down.

Auto installation

There was a small misunderstanding. I actually didn’t mean to reference auto installation in the context of development. On the contrary! I actually meant during deployment.

We want our packages to be built into our docker images at build time, but then be installed automatically at docker run time. Seems like what auto_install is for, no?

To achieve this however we had to do a few things:

  1. Run Package.auto_install every time the container starts (see here)
  2. Run rake assets:precompile every time the container starts (see here) – the addons that make UI changes require this, and running assets:precompile during the container build process is not sufficient as the package is not yet installed!
  3. Apply a small patch to Package.auto_install to make it upgrade already installed packages, and ignoring existing packages that are already installed. Otherwise, when the container starts the second time, it will fail to install the package with an error that the Package is already installed.

This lets us create a gitlab pipeline that builds the docker image when an addon project changes, bundle that addon into the image and make it available for use immediately. That project is private atm, but the Makefile looks like this https://gitlab.com/snippets/1943518. You can see we had to do some renaming to rename the packages from .szpm to .zpm.

In the end though its ok :slight_smile: we can rename them without issue, so no worries about the PR.

But what is the difference between .szpm and .zpm?

4

We solved this by using an initializer like @thorsteneckel suggested:

Rails.application.config.after_initialize do
  Ticket::Article.add_observer Observer::Ticket::Article::CommunicateSigarillo.instance
end

source

We solved this by simply editing public/assets/images/icons.svg at runtime, in yup, you guessed it, an initializer:

  icon = File.read("public/assets/images/icons/sigarillo.svg")
  doc = File.open("public/assets/images/icons.svg") { |f| Nokogiri::XML(f) }
  if !doc.at_css('#icon-signal')
    doc.at('svg').add_child(icon)
    Rails.logger.debug "sigarillo icon added to icon set"
  else
    Rails.logger.debug "sigarillo icon already in icon set"
  end
  File.write("public/assets/images/icons.svg", doc.to_xml)

source

1 Like