2.06 - Upgrading Squash TM

WARNING: before any modification, stop Squash TM

Users are provided with .sql upgrade scripts in each Squash TM release (see in /database-script ).

For Squash TM 1.13 and 1.14, the upgrade steps were slightly modified.

For Squash TM 1.15, logging libraries were upgraded, the logging configuration file and startup script must be updated.

From Squash TM 1.17 on, it is IMPERATIVE to use Java 8, Java 7 is not supported anymore.

The upgrade database steps are described below, depending on the origin and destination versions.

From Squash TM 1.20, the indexing is removed. Thus, this option is not available anymore.


Upgrading Squash TM from 1.19.X to 1.20+

 

In order to upgrade to version 1.20, the usual procedure applies (binaries replacement, execution of the SQL scripts that are needed…).

Nonetheless, the indexing was removed in version 1.20. This appendix details the impact of this removal.


        1. Removal of luceneindexes (optional)

As the indexing is removed, the “luceneindexes” directory is no longer of use. Here are the steps to be carried out:  

  1. Remove the “luceneindexes” directory

To find it:

  • Open the startup script (startup.bat for Windows, startup.sh for Linux, squash-tm for Debian and Redhat
  • Find "LUCENE_DIR": the location will be mentioned

2.       Remove the luceneindexes mentions in the startup script

If the startup script was not replaced when upgrading, it is necessary to remove the following elements in the script:

  • The line: LUCENE_DIR=%SQUASH-TM_HOME%/luceneindexes
  • The parameter --spring.jpa.properties.hibernate.search.default.indexBase=%LUCENE_DIR%, which can be found on the parameter definition line of java process.

3.       Remove the luceneindexes mentions from the Squash TM configuration file

If the configuration file (squash.tm.cfg.properties) in the Squash TM “conf” directory, was not replaced when upgrading, it is required to remove the following element in this file:

  • The property:

spring.jpa.properties.hibernate.search.default.indexBase=${squash.path.root}/luceneindexes

 

        2. Warning when the script is upgraded

 

When the script is upgraded to the version 1.20, warnings may be displayed.

These warnings are for information purpose only and will not have any impact on the application. They are the result of the indexing removal and the FULLTEXT index addition for search.

For MySQL database:

The following message is displayed: WARNING: 1

By clicking on SHOW WARNINGS, the following message is displayed: "InnoDB rebuilding table to add column FTS_DOC_ID"

 

For PostgresSQL database:

For each word having more than 2047 characters (for instance, images that are directly pasted in the “rich text” fields), the following messages are displayed:

“NOTICE: the word is too long to be indexed”

“DETAIL: the words having more than 2047 characters are ignored”


Upgrading Squash TM from 1.18- to 1.19+

In Squash TM 1.19, several internal composant have been updated. When upgrading to Squash TM 1.19, it is necessary to:
  • delete the content of the lucenceindexes directory
  • update the startup script
Here is the upgrade procedure:
  1. Backup your database
  2. Execute in growing order the files of which xxx is higher than the release number to be updated, and where the DBType is the one of your database ('DBType-upgrade-to-xxx.sql').
  3. Follow the updated Squash installation instruction (see 2 - Installation of Squash TM) 
  4. Update the startup configuration file for the new release (see below 1.Deleting the content of the luceneindexes directory)
  5. If modified, copy and paste the former conf/ file to the new file /conf/squash.tm.cfg.properties
  6. Delete the content of the luceneindexes directory (see below 2. Updating the startup script)

   1. Deleting the content of the luceneindexes directory

  1. Access the luceneindexes directory. To locate it:
    • open the startup script (startup.bat for Windows, startup.sh for Linux, squash-tm for Debian and Redhat)
    • find "LUCENE_DIR" : the location will be mentioned
  2. Delete the content of the lucenindexes directory

    2. Updating the startup script

            a) Option 1: Replace the startup script

~SQUASH: Squash TM install repository

If Squash TM is deployed on a Windows environment:

  1. Save the file ~SQUASH/bin/startup.bat
  2. Replace this file with the one in Squash TM 1.19 distribution: /bin/startup.bat
  3. Copy the database configuration and every other modifications from the saved file to the new file ~SQUASH/bin/startup.bat

If Squash TM is deployed on a Linux or MacOS environment:

  1. Save the file ~SQUASH/bin/startup.sh
  2. Replace this file with the one in Squash TM 1.15 distribution: /bin/startup.sh
  3. Copy the database configuration and every other modifications from the saved file to the new file ~SQUASH/bin/startup.sh

            b) Option 2: Edit the startup script

If Squash TM is deployed on a Windows environment:

  1. Save the file ~SQUASH/bin/startup.bat to restore if necessary
  2. In the file ~SQUASH/bin/startup.bat, find the following line: 

  3. set DAEMON_ARGS=%JAVA_ARGS% -Djava.io.tmpdir=%TMP_DIR% -Dlogging.dir=%LOG_DIR% -jar %JAR_NAME% --spring.profiles.active=%DB_TYPE% --spring.config.location=file:%CONF_DIR%/ --spring.config.name=application,squash.tm.cfg --squash.path.root=%SQUASH-TM_HOME% --squash.path.bundles-path=%BUNDLES_DIR% --squash.path.plugins-path=%PLUGINS_DIR% --server.port=%HTTP_PORT% --server.tomcat.basedir=%TOMCAT_HOME% --hibernate.search.default.indexBase=%LUCENE_DIR% --logging.config=%CONF_DIR%/log4j2.xml

  4. Edit the properties: 

     Former property
     New property
     --spring.config.location=file:%CONF_DIR%/ --spring.config.additional-location=file:%CONF_DIR%/
     --hibernate.search.default.indexBase=%LUCENE_DIR% --spring.jpa.properties.hibernate.search.default.indexBase=%LUCENE_DIR%


If Squash TM is deployed on a Linux or MacOS environment:

  1. Save the file ~SQUASH/bin/startup.sh to restore if necessary
  2. In the file ~SQUASH/bin/startup.sh, find the following line: 

  3. set DAEMON_ARGS=%JAVA_ARGS% -Djava.io.tmpdir=%TMP_DIR% -Dlogging.dir=%LOG_DIR% -jar %JAR_NAME% --spring.profiles.active=%DB_TYPE% --spring.config.location=file:%CONF_DIR%/ --spring.config.name=application,squash.tm.cfg --squash.path.root=%SQUASH-TM_HOME% --squash.path.bundles-path=%BUNDLES_DIR% --squash.path.plugins-path=%PLUGINS_DIR% --server.port=%HTTP_PORT% --server.tomcat.basedir=%TOMCAT_HOME% --hibernate.search.default.indexBase=%LUCENE_DIR% --logging.config=%CONF_DIR%/log4j2.xml

  4. Edit the properties: 

     Former property
     New property
     --spring.config.location=file:%CONF_DIR%/ --spring.config.additional-location=file:%CONF_DIR%/
     --hibernate.search.default.indexBase=%LUCENE_DIR% --spring.jpa.properties.hibernate.search.default.indexBase=%LUCENE_DIR%

    

    3. Debian Upgrade

If squash-tm has been installed from squashtest repo for Debian (i.e installed with apt commands):

    1. Backup your database
    2. apt upgrade squash-tm
    If your squashtm database has been automatically configured with dbconfig-common, then dbconfig-common will ask you if you wish it to automatically upgrade your database scheme. Please decline it.
    3. Upgrade your database scheme:
    cd /usr/share/dbconfig-common/data/squash-tm/upgrade/{mysql/pgsql}
        run the needed sql script (if your previous squash-tm version is 1.18.+, run the 1.19.0 script against your squashtm database ; if your previous version is 1.17.+, run 1.18.0 script against your database and then run 1.19.0 script)
    4. Delete the content of your squash-tm cache:
    cd /var/cache/squash-tm/lucene-indexes 
    rm -rf *

    5. Check your init script (/etc/init.d/squash-tm) is using the new version, i.e new parameter names for java arguments (See below 2. Updating the startup script)
    6. Restart squash-tm

Here is an example of upgrading squash-tm from 1.17.4 to 1.19.1:
root@example: ~# apt policy squash-tm
squash-tm:
  Installed: 1.17.4.RELEASE-8
  Candidate: 1.19.1.RELEASE-8
  Version table:
     1.19.1.RELEASE-8 500
        500 http://repo.squashtest.org/debian stretch/main amd64 Packages
 *** 1.17.4.RELEASE-8 100
        100 /var/lib/dpkg/status

## OK, a new squash-tm version is available
## let's backup our database:
root@example: ~# mysqldump -u squash-tm -p squashtm > squash_dump_$(date +%F).sql
Enter password:
root@example: ~# ls -lh
total 2.1G
-rw-r--r-- 1 root root 1.1K Jan 15 15:11 backup.crt
-rw-r--r-- 1 root root 1.7K Jan 15 15:10 backup.key
-rwxr-xr-x 1 root root  614 Feb  8 10:49 local_deploy_ssl.sh
-rw-r--r-- 1 root root 2.1G Feb 20 15:22 squash_dump_2019-02-20.sql

## let's upgrade squash-tm now:
root@example: ~# apt upgrade -y squash-tm

## let's upgrade our database:
root@example: ~# cd /usr/share/dbconfig-common/data/squash-tm/upgrade/mysql/
root@example /usr/share/dbconfig-common/data/squash-tm/upgrade/mysql# mysql -u squash-tm -p squashtm < 1.18.0
Enter password:
root@example /usr/share/dbconfig-common/data/squash-tm/upgrade/mysql# mysql -u squash-tm -p squashtm < 1.19.0
Enter password:

## clear squash-tm cache:
root@example /usr/share/dbconfig-common/data/squash-tm/upgrade/mysql# cd /var/cache/squash-tm/lucene-indexes/
root@example /var/cache/squash-tm/lucene-indexes# rm -rf *

## restart squash:
root@example: ~# systemctl restart squash-tm

## make sure everything went fine
root@example: ~# tail -f /var/log/squash-tm/squash-tm.log
# until a see a line like:
[19-02-20 15:36:32.529] SquashTM - 5360  INFO [main] [] --- org.squashtest.tm.SquashTm: Started SquashTm in 41.821 seconds (JVM running for 42.751)

Upgrading Squash TM from 1.16- to 1.18.5

  1. Backup your database
  2. Execute in growing order the files of which xxx is higher than the release number to be updated, and where the DBType is the one of your database ('DBType-upgrade-to-xxx.sql').
  3. Follow the updated Squash installation instruction (see 2 - Installation of Squash TM)
  4. Copy and paste the former startup configuration file to the new release
  5. If modified, copy and paste the former conf/ file to the new file /conf/squash.tm.cfg.properties

Upgrading Squash TM from 1.14.2- to 1.15+

In Squash TM 1.15, logging library has been upgraded from Log4j to Log4j2. Logging configuration file must be updated.

The procedure has two steps:

  • Logging configuration file migration
  • Squash TM startup script migration

The usual procedure must be followed as well (execution of SQL scripts to upgrade the database, substitution of configuration files if modified, etc. See "Upgrade Squash TM from 1.13+ to 1.13+" section below).

~SQUASH: Squash TM install repository

    1. Updating the logging configuration file

  1. Copy conf/log4j2.xml from Squash TM 1.15 distrubution to ~SQUASH/conf/log4j2.xml
  2. If the logging configuration is customized, it must be migrated from Log4j to Log4j2 format (see https://logging.apache.org/log4j/2.x/manual/configuration.html#XML)
  3. Delete the former configuration file: conf/log4j.properties

NB: Log4j2 can be configured with several formats. Squash TM uses XML format by default. You can pick another format for Squash TM logging configuration, but if you change the configuration file type, do not forget to type in the correct name and extension in the Squash TM run command (see "Update the run script").

    2. Updating the startup script

        a) Option 1: Replace the startup script

If Squash TM is deployed on a Windows environment:

  1. Save the file ~SQUASH/bin/startup.bat
  2. Replace this file with the one in Squash TM 1.15 distribution: /bin/startup.bat
  3. Copy the database configuration and every other modifications from the saved file to the new file ~SQUASH/bin/startup.bat

If Squash TM is deployed on a Linux or MacOS environment:

  1. Save the file ~SQUASH/bin/startup.sh
  2. Replace this file with the one in Squash TM 1.15 distribution: /bin/startup.sh
  3. Copy the database configuration and every other modifications from the saved file to the new file ~SQUASH/bin/startup.sh

        b) Option 2: Edit the startup script

If Squash TM is deployed on a Windows environment:

  1. Save the file ~SQUASH/bin/startup.bat to restore if necessary
  2. In the file ~SQUASH/bin/startup.bat, find the following line:

  3. set DAEMON_ARGS=%JAVA_ARGS% -Djava.io.tmpdir=%TMP_DIR% -Dlogging.dir=%LOG_DIR% -jar %JAR_NAME% --spring.profiles.active=%DB_TYPE% --spring.config.location=file:%CONF_DIR%/squash.tm.cfg.properties --squash.path.root=%SQUASH-TM_HOME% --squash.path.bundles-path=%BUNDLES_DIR% --squash.path.plugins-path=%PLUGINS_DIR% --server.port=%HTTP_PORT% --server.tomcat.basedir=%TOMCAT_HOME% --hibernate.search.default.indexBase=%LUCENE_DIR% --logging.config=%CONF_DIR%/log4j.properties

  4. Edit the property:

     Former property
     New property
     --logging.config=%CONF_DIR%/log4j.properties  --logging.config=%CONF_DIR%/log4j2.xml


If Squash TM is deployed on a Linux or MacOS environment:

  1. Save the file ~SQUASH/bin/startup.sh to restore if necessary
  2. In the file ~SQUASH/bin/startup.sh, find the following line:

    set DAEMON_ARGS=%JAVA_ARGS% -Djava.io.tmpdir=%TMP_DIR% -Dlogging.dir=%LOG_DIR% -jar %JAR_NAME% --spring.profiles.active=%DB_TYPE% --spring.config.location=file:%CONF_DIR%/squash.tm.cfg.properties --squash.path.root=%SQUASH-TM_HOME% --squash.path.bundles-path=%BUNDLES_DIR% --squash.path.plugins-path=%PLUGINS_DIR% --server.port=%HTTP_PORT% --server.tomcat.basedir=%TOMCAT_HOME% --hibernate.search.default.indexBase=%LUCENE_DIR% --logging.config=%CONF_DIR%/log4j.properties

  3. Edit the property:

     Former property
     New property
     --logging.config=%CONF_DIR%/log4j.properties  --logging.config=%CONF_DIR%/log4j2.xml


Upgrading Squash TM from 1.12.6- to 1.13+

In TM 1.13, we switched from an OSGi container to a spring-boot based microservice container. From now on, the embedded servlet engine will be Tomcat instead of Jetty, and it will offer more configuration possibilites. To upgrade a SquashTM installation from a version prior to 1.13, to a version after 1.13, please follow these steps :
  1. Backup your database
  2. Run in the xxx growing order every 'DBType-upgrade-to-xxx.sql' script, of which number xxx is higher than the release number to be updated, and the DBType is the type of your database.
  3. Follow the updated Squash installation instruction (see 2 - Installation of Squash TM) 
  4. All plug-ins need to be updated to TM 1.13 compatible versions. Every plug-in is now installed by dropping it in the ~SQUASH/plugins folder. Plug-ins installed in ~SQUASH/bundles will not be picked by Squash TM.
  5. Migrating startup shell scripts (startup.bat for windows, startup.sh for linux)
    You need to use the new shell scripts located in ~SQUASH/bin. In order to configure your application, you need to refer to the variables table below. All variable featuring "unchanged" in the 1.13+ column can be copied from your previous startup.bat/sh. 
    All relative paths are relative to the shell location of the script i.e. ~SQUASH/bin. Unlisted variables should not be modified anyway. Please note that you must configure the variable DB_TYPE, which was not existing before 1.13.

     Variable Name  up to 1.12.X  1.13 +
     JETTY_HOME  Jetty's work folder  Removed
     TOMCAT_HOME  n/a  Tomcat's work folder
     CACHE_DIR  Path to felix-cache  Removed
     LUCENE_DIR  Lucene indexes folder  unchanged
     TMP_DIR  Temp files folder  unchanged
     DB_URL  JDBC url of the database  unchanged
     DB_USERNAME  database user  unchanged
     DB_PASSWORD  database password  unchanged
     DB_DIALECT  Hibernate database dialect  removed
     DB_DRIVER  JDBC driver  removed
     DB_TYPE  n/a  one of : h2,mysql,postgresql

  6. Migrating configuration files

    Only 2 configuration files are required in the ~SQUASH/conf folder : squash.tm.cfg.properties and log4j.properties
    These former configuration files are no longer used and can safely be removed : 

    felix.config.properties and felix.system.properties

    If you configured Squath TM using properties from any other ~SQUASH/conf/services/*.properties file, you should migrate these properties in ~SQUASH/conf/squash.tm.cfg.properties
     according to the following table. Properties which are not listed below should not be migrated. These were not configurable properties : they were not to be changed in any way.

     Former file name  Former property name  New property name  Note
     org.squashtest.csp.core.auth.config  authentication.provider  unchanged  See plugin documentation for additional properties
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.driverClassName  removed  Handled through DB_TYPE in shell script
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.url  spring.datasource.url  Not required when configured in shell script
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.username  spring.datasource.username  Not required when configured in shell script
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.password  spring.datasource.password  Not required when configured in shell script
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.validationQuery  spring.datasource.validation-query  Not required, sensible defaults applied
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.maxActive  spring.datasource.max-active  Not required, defaults to 10
     org.squashtest.csp.core.datasource.jdbc.config  datasource.db.initialSize  spring.datasource.initial-size  Not required, defaults to 2
     org.squashtest.csp.tm.hibernate.config  hibernate.dialect  removed  Handled through DB_TYPE in shell script
     squash.tm.cfg.properties  report.criteria.project.multiselect  unchanged  
     squash.tm.cfg.properties  squashtm.bugtracker.timeout  unchanged  
     tm.testautomation.conf  tm.test.automation.pollinterval.millis  unchanged  
     tm.testautomation.conf  tm.test.automation.server.callbackurl  unchanged  

Updating Squash TM from version 1.12.6- to version 1.12.6-

To update the database, you will have to run all scripts 'bdd-upgrade-to-xxx.sql' where bdd is the type of your database, and xxx is higher than the version to update from (the scripts have to be run in the xxx growing order).

For example : for an update from 1.0.2 to 1.1.1 on a MySQL database, you need to run in this order :
  • mysql-upgrade-to-1.1.0.RELEASE.sql
  • mysql-upgrade-to-1.1.1.sql

BEWARE

Even though our upgrade scripts are thoroughly tested before each release, we strongly recommend that you back up your database before using them.

To complete the update, you must index your datas :
  1. Login as administrator in Squash
  2. Go to administration workspace
  3. Click on [Index management]
  4. Click on [Index everything]
Your database is now ready.

H2 database

In order to retrieve your data, you need a tool to connect to your h2 database. For example, razor SQL will enable you to easily connect with your database (select H2 , then H2 embedded).
Once you are logged in (squash>squashtm>data>squash-tm.h2.db), you are able to run the migration scripts of the database (squash>squashtm>database-scripts> h2-upgrade-to-1.1.1.RELEASE.sql). 

FOCUS : Recommended

The embeded h2 database of Squash is not optimal for production maybe you should consider migrating to SQL (as seen here)



Property files

Once the application is installed on your disk, you might want to import your previous server configuration (in the conf/services directory). Copying and pasting would be sufficient, except if changes were made, or if properties were added in the new version. In the latter case, you will be notified about it.

Clearing Cache

Finally, you need to clear the browser cache for all the client machine for the domain hosting the instance of Squash TM. You can usually do this by pressing Ctrl + Shift + Suppr, then selecting the cached Internet files. On the server, it may also help to remove the 'felix-cache' directory. Its location may vary, depending on the system it runs on.




DEBIAN Upgrade

1. Stop Squash
2. Upgrade squash :
sudo apt-get upgrage squash-tm




REDHAT Upgrade

1. Stop Squash
2. Upgrade your database (see above)
3. Upgrade Squash :
sudo yum upgrade squash-tm