Upgrading notes

Note

See also "Upgrading Sympa" for general upgrading instruction.

Upgrading from Sympa 6.2.x or earlier

After release of 6.2, several changes on templates are made. If you have customized templates with earlier version, you should check if web interface will work correctly after upgrading, and reapply customization to new templates as necessity.

Following subsections describe changes by particular versions of 6.2.x. If you are planning to upgrade from version prior to 6.2, see also sections below.

From version prior to 6.2.72

The dkim authentication method for scenarios was obsoleted. Now it is a synonym of smtp method. See also the documentation.
The antispam_feature parameter was deprecated. Instead, choose appropriate scenario with spam_status parameter. In particular, if you wish to disable this feature, set "spam_status none".
The Command line tools under $SCRIPTDIR either were deprecated or were integrated into the sympa command line tool. See issue #1386 for details.

From version prior to 6.2.70

sympa_wizard.pl was deprecated. About alternatives see issue #508.
For the users of Systemd: The unit files described in Using Systemd socket have been added to the source distribution. Some packagers (at least Debian and RPM) have adopted these and no longer use the setuid wrappers for the FastCGI servers. Also, when using these unit files, stopping the web service requires stopping the wwsympa.socket unit instead of the wwsympa.service unit (same for sympasoap).

From version prior to 6.2.68

sympa.pl: The option --import is now deprecated. Instead, use the new option --add. See sympa.pl(1) about usage of the new option.

From version prior to 6.2.62

Perl version 5.16.0 or later will be supported.
Note
- As of Sympa 6.2.61b, support for Perl 5.14.x or earlier has been dropped.

From version prior to 6.2.60

Personalization (also known as "merge feature") is now subject to the following restrictions:
- Allowed only for messages posted from the web interface
- Applied only to the header and the footer of the message
The restrictions can be configured with the new personalization list parameter. Listmasters are advised to review the current usage of template variables and adjust the configuration accordingly.
Now the setuid wrappers may be disabled, if installation process allows.
- The new options for configure script --disable-setuid and so on may be used to disable setuid features. See the manual for details.
- The new parameter aliases_wrapper in sympa.conf may be off not to use setuid program sympa_newaliases-wrapper and it may be removed.
Packagers are encouraged to provide configuration not using setuid wrappers as possible. See also #943 and related issues/PRs.

From version prior to 6.2.56

If you have set http_host parameter in sympa.conf or robot.conf, you may have to change it.
- If the value of http_host is identical to the host part of wwsympa_url parameter, it may simply be removed. For example,
```
wwsympa_url       http://web .example.org/sympa
http_host         web.example.org
```
  may be changed to
```
wwsympa_url       http://web.example.org/sympa
```
- Otherwise, you have to replace it with appropriate wwsympa_url_local parameter. For example,
```
wwsympa_url       http://web.example.org/sympa
http_host         backend.example.org
```
  should be modified as
```
wwsympa_url       http://web.example.org/sympa
wwsympa_url_local http://backend.example.org/sympa
```

From version prior to 6.2.54

If you are using the family_signoff link (the URL link in message footer that allows unsubscribing from all the lists in a family), you have to modify message_footer.tt2 of the family according to new format (See "Family unsubscription" for details).

From version prior to 6.2.50

Some scenarios and list creation templates for "intranet" use cases were made optional: They have been moved into samples/.

If you have to use following scenario files, copy those files in samples/intranet/scenari/ to appropriate scenari/ directory:
- archive_web_access.intranet
- create_list.intranet
- review.intranet
- send.intranet
- send.intranetorprivate
- subscribe.intranet
- subscribe.intranetorowner
- visibility.intranet
If you have to use these files in a list creation template, copy those files in samples/intranet/create_list_templates to a subdirectory of appropriate create_list_template/ directory:
- intranet/comment.tt2
- intranet/config.tt2

From version prior to 6.2.48

Inclusion feature of members, owners or moderators has been reconstructed. Slightly long time can be spent to refresh inclusion of users entirely at the first time just after upgrade.
Perl version 5.10.1 or later will be supported.
Note
- As of Sympa 6.2.45b, support for Perl 5.10.0 or earlier has been dropped.

From version prior to 6.2.44

WWSympa: TLS client authentication: Now it gets rfc822Name in X.509v3 subjectAltName, otherwise emailAddress attribute in subject DN. Note that earlier efforts getting attribute such as MAIL, Email in subject DN are no longer supported.

From versions prior to 6.2.42

Authorization schearios: The "default" scenario files named *.default (regular file or symbolic link) are no longer available: Default list scenariios have to be specified by parameters in robot.conf or sympa.conf.

For details on parameters see "Default privileges for the lists" in sympa.conf(5) manual page. Previous default settings using symbolic links are automatically migrated during upgrading process. However you should review the changes in sympa.conf (and robot.conf).
WWSympa and SympaSOAP:
- If a virtual domain setting does not have auth.conf, crawlers_detection.conf or trusted_applications.conf while it is there in $SYSCONFDIR, the latter will be used. Previously in such case, the latter was ignored and only built-in authnetication was enabled.
- Built-in authantication: RC4 reversible encryption of password storage in database using Crypt::CipherSaber was dropped. If you have been using it, run sympa upgrade password (if you are upgrading to Sympa 6.2.70 or earlier, upgrade_sympa_passowrd.pl) to rehash passwords stored in database.
  Note
  - Though it is not forced, it is recommended to upgrade password storage format using bcrypt, more secure hash function. See "Upgrading password storage on earlier version" for details.
- LDAP authentication: Now entry of authenticating user is retrieved by the LDAP account specified by bind_dn parameter. Previously, the second search operation to retrieve user entry was performed under the privilege of the user of their own.
- Format of session cookie was changed. Even if you have been used web interface with Sympa 6.2 or later, all users may have to login again after upgrade.

From versions prior to 6.2.38

If you have used Oracle Database, review db_* parameters in sympa.conf:
- If you want to continue using SID (only method supported before) and you have not set db_host parameter explicitly, add a line
```
db_host localhost
```
  to sympa.conf.
- Or, you may choose the other methods.
For details see the instruction.

From versions prior to 6.2.34

host list parameter was deprecated. If you have used this parameter:
1. Create new domain with the same name as value of host list parameter (Note: replace $SYSCONFDIR, $EXPLDIR and <host> below):
```
# mkdir -m 755 $SYSCONFDIR/<host>
# touch $SYSCONFDIR/<host>/robot.conf
# chown -R sympa:sympa $SYSCONFDIR/<host>
# mkdir -m 750 $EXPLDIR/<host>
# chown sympa:sympa $EXPLDIR/<host>
```
  And rename list listname@domain to listname@host, using web interface or sympa.pl command line tool.
2. Also, you may have to check list_aliases.tt2 and may have to regenerate alias file after upgrading as:
```
# sympa.pl --make_alias_file --robot <mail domain>
# sympa_newaliases.pl --domain <mail domain>
```
Note that you are recommended to back up database and older alias files in advance.
[conf->host] in scenario was depreceated. If you have used it, replace it with [domain].
If you managed multiple domains and used web interface, wwsympa_url parameter in each robot.conf file is now mandatory. Though it will be automatically added during upgrading process, if you used HTTPS protocol, you may have to edit value of wwsympa_url parameter in each robot.conf file.

From versions prior to 6.2.30

If sympa.pl --upgrade crashes due to undefined variable $pictures_dir, remove following files and retry:
- sympa.conf.bin placed in the same directory as sympa.conf file.
- robot.conf.bin placed in the same directory as each robot.conf file, if any.

From versions prior to 6.2.28

Renamed scenarios

access_web_archive.* scenarios are renamed to archive_web_access.*. If you have customized any of these scenarios, you have to rename them under config directory.

Changes on `css_path` and `css_url` parameters

css_url and css_path parameters in robot.conf are no longer available: Those in sympa.conf (if any) are used.

If you have specified css_path parameter by each robot.conf, you have to move each directory to subdirectory named by domain under directory specified by css_path directory in sympa.conf (by default $CSSDIR).

New `configure` options and parameters

If you have used none of --with-staticdir configure option, static_content_path parameter nor static_content_url parameter, no changes described below are required.
If you have built Sympa from source and have specified --with-staticdir configure option, you might want to specify --with-cssdir and --with-picturesdir also.
- With earlier version:
```
$ ./configure --with-staticdir=DIR (...)
```
- With recent version:
```
$ ./configure --with-staticdir=DIR --with-cssdir=DIR/css --with-picturesdir=DIR/pictures (...)
```
If you have specified static_content_path parameter in sympa.conf, you might want to specify css_path (if you have not specified it) and pictures_path also.
- With earlier version:
```
static_content_path DIR
```
- With recent version:
```
static_content_path DIR
css_path            DIR/css
pictures_path       DIR/pictures
```
If you have specified static_content_url parameter in sympa.conf, you might want to specify css_url (if you have not specified it) and pictures_url also.
- With earlier version:
```
static_content_url PATH
```
- With recent version:
```
static_content_url PATH
css_url            PATH/css
pictures_url       PATH/pictures
```

New password hash format

It is not forced, but it is recommended to upgrade password storage format of web interface using bcrypt, more secure hash function. See "Upgrading password storage on earlier version" for details.

From versions prior to 6.2.24

Data sources: "%" sign in SQL data source no longer need escaping by duplicating it, i.e. "%%". Administrators on the sites allowing SQL data sources are recommended to check data source settings and modify them as necessity.
WWSympa: FastCGI support became mandatory. CGI mode was deprecated. See "Configure HTTP server for details about configuration.

From versions prior to 6.2.18

Especially on 6.2.18, web templates for shared document repository and list configuration edit form broke backward compatibility in exchange for bug fixes.

Upgrading from Sympa 6.1.x or earlier

Several changes in Sympa 6.2 require to perform specific operations when upgrading. Here is the ordered list of operations to perform.

[Source distribution only] Check the options for configure script.
- As some Sympa binaries have been renamed, your sympa init script have to be updated. Please make sure your configure options have been correctly set, so that this script ends up in the correct location. See "Run configure script" for details.
- Options --with-sendmail_aliases and --with-virtual_aliases were deprecated. Use --with-aliases_file instead. Option --with-postmap_arg was removed.
[Source distribution only] Run additional commands after make install:

First, update dependent modules: See "Install dependent modules".

Next, run these commands to upgrade configuration and user data.
```
# sympa.pl --upgrade_config_location # move existing configs to /etc/sympa/ directory
# sympa.pl --upgrade                 # merge sympa.conf and wwsympa.conf
```

Commands after upgrade:

These are always needed you are using either binary distribution or source distribution.

If you are upgrading to Sympa 6.2.72 or later:

# sympa upgrade outgoing             # move messages stored in database to filesystem
# sympa upgrade incoming             # move messages sent through web interface to the new format

For details, read manual pages of sympa upgrade outgoing and sympa upgrade incoming.

If you are upgrading to Sympa 6.2.70 or earlier:

# upgrade_bulk_spool.pl              # move messages stored in database to filesystem
# upgrade_send_spool.pl              # move messages sent through web interface to the new format

For details, read manual pages of upgrade_bulk_spool.pl and upgrade_send_spool.pl.

For all your custom action templates, move them into $SYSCONFDIR/custom_actions directory.

After the process above, the following changes will have occured:

The sympa.conf is now located in a sympa/ subdirectory.
Content of wwsympa.conf will have been merged into sympa.conf. wwsympa.conf will no longer be used.
The sympa.conf remaining will have been stripped of any "color_*" parameters, to prevent you from having a messed up web interface.
- Any robot.conf will have had the same treatment as sympa.conf (copy, then stripped of color_* parameters).
Older sympa.conf will have need copied to "sympa.conf.upgrade.number".
Any customized "web_tt2" directory (either in $SYSCONFDIR/ or in $SYSCONFDIR/domain/) will have been renamed to "web_tt2.upgrade.number".

Indeed, as most of the Sympa templates have been changed for the new skin, your customizations could not be compliant to the new web interface. You should see how to reintroduce them to the new templates.

Additionally, you may have to fix up configuration manually:

New parameters:

process_archive controls archiving. The default is "off": To enable archiving, it must be set to "on" explicitly (in sympa.conf, robot.conf or each list config file).

If you have used virtualwrapper, it was replaced with new programs sympa_newaliases.pl and its wrapper: You may have to add new parameters aliases_program and optionally aliases_db_type to sympa.conf.
Renamed list parameters:

web_archive.access to archive.web_access; archive.access to archive.mail_access; web_archive.quota to archive.quota; web_archive.max_month to archive.max_month.
Deprecated list parameter:

user_data_source. All subscribers are stored in database. The subscribers file in list directory will no longer be imported.
Administrators are strongly recommended to check new configuration. Customized scenarios, list creation templeates and edit_list.conf will never be upgraded automatically.

Upgrading from Sympa 5.4.x or earlier

Since there are big changes after 5.4.x, we recommend that recent version of Sympa will be installed into new machine, or at least will be installed under separate directory, keeping installation and data of earlier version.

Stop services of earlier version, for example doing:
```
# /etc/rc.d/init.d/sympa stop
```
Then back up everything.

Note that you should also backup init script of earlier version, because it may be overwritten by installing recent version (if you will install recent version to the same machine).
Install recent version of Sympa.
- See "Installing Sympa" to install recent version.
Copy configuration files and database:
- Copy configuration files sympa.conf and wwsympa.conf in earlier version to recent version. Then edit them to fix up configuration (including name of database below).
- Queued messages in spools of earlier version should be copied into the new location. Their formats will be changed later.
- Restore entire database content as a new database (e.g. naming it as "sympa6" instead of "sympa"). Because, succeeding process will upgrade content and structure of database and those changes are not recoverable.
  Note
  - On this occation, you might want to check if database schema support Unicode-aware character set, e.g. utf8, UNICODE, AL32UTF8. Even if it does not, Sympa will work, however it is desirable. To know how to convert your database to be Unicode-aware, please consult to documentation of database server.
Upgrade configuration and data:
- With recent version of Sympa, run, for example:
```
# sympa.pl --upgrade --from=5.4.7
```
  Note that 5.4.7 above must be replaced with the version number you are updating from. Doing this, configuration and database structure will be upgraded.
- Upgrade format of web interface password in the database. See "Upgrading password storage on earlier version".
- Check customizations of templates, scenarios and list creation templates on earlier version, and reapply them to recent version if possible.
Start services of recent version (see "Starting services"), then check if everything goes well.

Upgrading from Sympa prior to 5.0

(Not yet written)