Merge commit for 4.0.2

ChangeLog

@@ -1,3 +1,35 @@
+Tue, 12 Mar 2024 11:33:49 +0100 IvozProvider Team <[email protected]>
+
+    * IvozProvider 4.0.2 released
+
+    * Portals
+        - Fixed multiple select options only displaying 30 entries
+        - Fixed a bug that removed Active Calls after two minutes
+        - Fixed a bug that prevented some Active Calls to be removed when finished
+        - Fixed display issues when browser text language was not English or Spanish
+        - Fixed time and datetime selectors for Firefox browser
+        - Added Invoice scheduler last execution status icon
+        - Updated dashboard icons, colors and texts
+        - Fixed multiple translations issues
+        - Theme based web titles
+
+    * Provisioning
+        - Added missing Terminal provisioning microservice package
+        - Removed not allowed PHP functions from default templates
+
+    * Proxies
+        - Fixed SUBSCRIBE messages socket selection and allowed Events
+        - Handle SUBSCRIBE in custom route to skip INVITE-only logics
+        - Fixed cgrates-reload script to work outside a data profile machine
+        - Application server failover logic is not longer triggered when a 404 is received
+
+    * Billing
+        - Fixed a bug displaying Client data in Invoices
+
+    * Other
+        - Reviewed documentation
+        - Added upgrade notes for tempest release
+
 Wed, 28 Feb 2024 14:21:59 +0100 IvozProvider Team <[email protected]>
 
     * IvozProvider 4.0.1 released

README.md

@@ -51,7 +51,7 @@ If you want to test an [standalone](https://irontec.github.io/ivozprovider/en/ar
 | oldoldstable (oasis 1.7)   |     [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-1.7.1-oasis-amd64.iso)     | [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-1.7.1-oasis-i386.iso)|
 | oldstable (artemis 2.23.0) | [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-2.23~2.23.0-artemis-amd64.iso) | |
 | stable (halliday 3.4.1)    | [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-3.4~3.4.1-halliday-amd64.iso)  | |
-| testing (tempest 4.0.1)    |  [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-4.0~4.0.0-tempest-amd64.iso)  | |
+| testing (tempest 4.0.2)    |  [![iso http](web/admin/public/images/iso-http-green.png)](https://packages.irontec.com/isos/ivozprovider-4.0~4.0.0-tempest-amd64.iso)  | |
 
 
 You can read about differences between releases [here](https://github.com/irontec/ivozprovider/blob/bleeding/FAQ.md#what-release-should-i-use).

cgrates/scripts/cgrates-reload

@@ -16,6 +16,10 @@ ALLBRANDS=-1
 FLUSHALL=0
 RELOADDESTINATIONS=0
 
+# Database access settings
+export MYSQL_HOST="data.ivozprovider.local"
+export MYSQL_PWD="changeme"
+
 while getopts ":Fdb:" opt; do
   case $opt in
     F)

debian/changelog

@@ -1,4 +1,10 @@
-ivozprovider (4.0~4.0.1) UNRELEASED; urgency=medium
+ivozprovider (4.0~4.0.2) UNRELEASED; urgency=medium
+
+  * Version bump to 4.0.2
+
+ -- Irontec IvozProvider Team <[email protected]>  Mon, 11 Mar 2024 13:21:52 +0100
+
+ivozprovider (4.0~4.0.1) unstable; urgency=medium
 
   * Version bump to 4.0.1
 

debian/control

@@ -106,6 +106,7 @@ Depends: ivozprovider-profile-common (=${binary:Version}),
     ivozprovider-doc (=${binary:Version}),
     ivozprovider-async-workers (=${binary:Version}),
     ivozprovider-scheduler (=${binary:Version}),
+    ivozprovider-provision (=${binary:Version}),
     ivozprovider-realtime (=${binary:Version}),
     ivozprovider-router (=${binary:Version}),
     apache2,
@@ -305,6 +306,14 @@ Homepage: http://www.irontec.com
 Description: IVOZ Provider - Scheduled tasks runner - Symfony microservice
  .
 
+Package: ivozprovider-provision
+Architecture: all
+Priority: optional
+Depends: ivozprovider-common-library (=${binary:Version})
+Homepage: http://www.irontec.com
+Description: IVOZ Provider - Terminal provisioning - Symfony microservice
+ .
+
 Package: ivozprovider-balances
 Architecture: all
 Priority: optional

debian/ivozprovider-profile-proxy.postinst

@@ -37,6 +37,7 @@ function setup_mysql_access()
 
     # Replace password in configuration files
     sed -i -r "s#(password *= *).*#\1$MYSQL_PWD#" /etc/mysql/conf.d/kamailio.cnf
+    sed -i -r "s#(MYSQL_PWD *= *).*#\1\"$MYSQL_PWD\"#" /usr/bin/cgrates-reload
     sed -i -r "s#(\"db_password\": *)\".+\"#\1\"$MYSQL_PWD\"#" /etc/cgrates/cgrates.json
 }
 

debian/ivozprovider-provision.install

@@ -0,0 +1 @@
+microservices/provision /opt/irontec/ivozprovider/microservices/

debian/ivozprovider-provision.postinst

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+set -e
+
+#DEBHELPER#
+
+# Set current environment
+export APP_ENV=prod
+
+cd /opt/irontec/ivozprovider/microservices/provision
+
+# Set proper var permissions
+setfacl -dR -m u:www-data:rwX -m u:root:rwX var
+setfacl  -R -m u:www-data:rwX -m u:root:rwX var
+
+# Create project cache
+bin/console cache:clear --no-warmup -q -n
+
+:

debian/ivozprovider-provision.preinst

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+#DEBHELPER#
+
+# This is a temporal migration script to automatize changes implemented in #633
+if [ -d /opt/irontec/ivozprovider/microservices/provision/ ]; then
+    pushd /opt/irontec/ivozprovider/microservices/provision/
+        # Remove existing old cache
+        rm -fr var/cache
+        # Remove existing vendor (will be replaced with new package contents)
+        rm -fr vendor
+    popd
+fi
+
+:

debian/rules

@@ -52,6 +52,7 @@ build_microservices: build_library
 	cd microservices/recordings; composer --no-interaction --no-dev --no-progress --optimize-autoloader install
 	cd microservices/workers; composer --no-interaction --no-dev --no-progress --optimize-autoloader install
 	cd microservices/scheduler; composer --no-interaction --no-dev --no-progress --optimize-autoloader install
+	cd microservices/provision; composer --no-interaction --no-dev --no-progress --optimize-autoloader install
 	cd microservices/realtime; go mod download && go mod verify
 	cd microservices/realtime; go build -v -o server irontec.com/realtime/cmd/server
 	cd microservices/router; composer --no-interaction --no-dev --no-progress --optimize-autoloader install

doc/UPGRADE-4.0.md

@@ -0,0 +1,97 @@
+# Upgrading Instructions
+
+## Introduction
+
+This text is a task-list for migrating standlone installations from Halliday (3.X) to Tempest
+release (4.X). Although it's possible to upgrade directly an existing installation of Halliday
+to the new release, the Debian base system will jump from 11 to 12, so errors may appear during
+the upgrade.
+
+** If you want to avoid fixing package conflicts during upgrade, we highly recommend creating
+a new fresh installation of IvozProvider Tempest and load an existing database dump from previous
+release.**
+
+## Prerequisites
+
+Upgrade your installation to your latest available version (currently 3.4.1). This will ensure
+all database migrations are applied and the schema is up-to-date. This is required because Tempest
+migrations assume that schema status.
+
+Do a fresh IvozProvider tempest install from one of available methods.
+
+https://irontec.github.io/ivozprovider/en/tempest/basic_concepts/installation/debian_install.html
+
+
+## Database migration
+
+Create a database dump for your existing halliday (3.x) migration:
+
+```
+mysqldump -u root -p ivozprovider > /path/to/ivozprovider-halliday.sql
+```
+
+Move the database dump to a new tempest environment and reset its database
+
+Above commands will **DELETE** existing tempest database and create a new one from artemis data,
+applying all new migrations.
+
+```
+cd /opt/irontec/ivozprovider/schema
+bin/console doctrine:database:drop --force
+bin/console doctrine:database:create
+mysql -u root -p ivozprovider < /path/to/ivozprovider-artemis.sql
+bin/console doctrine:migrations:migrate -n
+```
+
+This will take from minutes to hours depending on the CDR tables size. If no error
+arises, then migrations will be executed properly.
+
+
+## Debian package configuration
+
+Following section assumes you are upgrading the system from latest halliday to tempest.
+This is not recommended and package conflicts will arise. It's preferable to do a new
+halliday fresh install and migrate database and storage contents.
+
+### Prepare APT sources for Debian Bullseye
+
+Add following contents into /etc/apt/sources.list.d/debian.list
+
+```
+# Debian Bookworm official repositories
+deb http://ftp.debian.org/debian bookworm main contrib non-free
+deb http://ftp.debian.org/debian bookworm-updates main contrib non-free
+deb http://ftp.debian.org/debian bookworm-backports main contrib non-free
+deb http://security.debian.org/debian-security bookworm-security main
+```
+
+## Prepare APT sources for IvozProvider Tempest
+
+Add following contents into /etc/apt/sources.list.d/ivozprovider.list
+
+```
+# Irontec IvozProvider tempest repository
+deb http://packages.irontec.com/debian tempest main extra
+deb http://packages.irontec.com/debian tayler main
+```
+
+## Update your installation to latest available version
+
+    apt-get update
+    apt-get upgrade
+
+Note: New database configurations assumes percona-server will be used. If you have trouble starting
+your database, install latest percona-server-server from tempest repositories.
+
+    apt-get install percona-server-server
+
+### Load CGRateS data
+
+Tempest uses CGRateS as billing engine, so all migrated data must be present in redis.
+
+Use following script to load all brands data to redis after database changes.
+
+```
+/usr/bin/cgrates-reload -b all -d
+```
+

doc/sphinx/administration_portal/brand/settings/notification_templates.rst

@@ -19,6 +19,8 @@ Brand administrators can configure the notifications sent by IvozProvider:
 
 - Email sent when max daily usage is reached
 
+- Email sent with access credentials
+
 .. hint:: When no custom notification is configured, default ones will be used
 
 Notifications are created in two steps: Create a notification type and add contents to the notification for each

doc/sphinx/administration_portal/client/residential/index.rst

@@ -22,11 +22,11 @@ associated with Residential clients included in IvozProvider:
     :titlesonly:
 
     residential_devices
-    ddis
-    external_call_filters
     residential_voicemails
     faxes
+    ddis
     rating_profiles
+    external_call_filters
     calls/index
 
 .. tip:: Check :ref:`Differences between retail and residential clients` to understand the difference between these two

doc/sphinx/administration_portal/client/residential/residential_devices.rst

@@ -39,6 +39,9 @@ Residential Devices Configuration
 
 These are the configurable settings of *Residential devices*:
 
+*******************
+Basic configuration
+*******************
     Name
         Name of the **residential device**. This name must be unique in the whole brand so 
         it's recommended to use some kind of sequential identifier. This will also be used
@@ -56,16 +59,30 @@ These are the configurable settings of *Residential devices*:
         If you choose 'Yes' here, you'll have to fill the protocol, address and
         port where this *residential device* can be contacted.
 
+    Multi Contact
+        Same SIP credentials can be configured in multiple SIP devices. In that case, all devices ring
+        simultaneously when receiving a call. Setting this toggle to 'No' limits this behaviour so that
+        only latest registered SIP device rings.
+
+************************
+Geographic configuration
+************************
     Language
         Locutions will be played in this language
 
     Numeric transformation
         Numeric transformation set that will be applied when communicating with this device.
 
+**********************
+Outgoing configuration
+**********************
     Fallback Outgoing DDI
         External calls from this *residential device* will be presented with this DDI, **unless
         the source presented matches a DDI belonging to the residential client**.
 
+**********************
+Advanced configuration
+**********************
     Allowed codec
         Like vPBX terminals, *residential devices* will talk only the selected codec.
 
@@ -78,22 +95,17 @@ These are the configurable settings of *Residential devices*:
         used in Contact header of registration will be used, as specified in SIP RFC (residential device name will be used
         for endpoints with direct connectivity). Defaults to 'No'.
 
-    Call waiting
-        Limits received calls when already handling this number of calls. Set 0 for unlimited.
-
     Enable T.38 passthrough
         If set to 'yes', this SIP endpoint must be a **T.38 capable fax sender/receiver**. IvozProvider
         will act as a T.38 gateway, bridging fax-calls of a T.38 capable carrier and a T.38 capable device.
 
+    Call waiting
+        Limits received calls when already handling this number of calls. Set 0 for unlimited.
+
     RTP Encryption
         If set to 'yes', call won't be established unless it's possible to encryption its audio. If set to 'no',
         audio won't be encrypted.
 
-    Multi Contact
-        Same SIP credentials can be configured in multiple SIP devices. In that case, all devices ring
-        simultaneously when receiving a call. Setting this toggle to 'No' limits this behaviour so that
-        only latest registered SIP device rings.
-
 .. tip:: Residential device can be contacted due to calls to several DDIs. *DDI In* setting allows remote SIP endpoint to
          know which number caused each call, setting that number as destination (R-URI and To headers). This way, residential
          device can handle calls differently.

doc/sphinx/administration_portal/client/vpbx/ddis.rst

@@ -4,12 +4,22 @@
 DDIs
 ####
 
+***********
+Number data
+***********
     Country
         The country of the new created DDI. Used for E164 standardization.
 
     DDI
         The number, without country code.
 
+    Display name
+        This value will be shown in the called terminals.
+
+    Language
+
+    Description
+
     Type
         Choose 'Inbound & outbound' for a normal DDI that can be used both as outgoing DDI and as incoming DDI
         from a :ref:`DDI Provider <DDI Providers>`. Choose 'Outbound only' for a DDI that won't reach us from a
@@ -19,15 +29,24 @@ DDIs
         The :ref:`DDI Provider <DDI Providers>` that provides this number. This relation has no functional purpose, it
         is just for DDI Provider <-> DDI navigation in some brand level sections.
 
+*************
+ Filters data
+*************
     External Call Filter
         Allows configuration based on Calendars and Schedulers as shown in
         :ref:`External Call Filters`. Leave empty if you don't need to apply any
         kind of filter.
 
-    Route
+*********************
+Routing configuration
+*********************
+    Route type
         A DDI can have different :ref:`treatments <routing_logics>`. For our
         current goal, set route to user and select *Alice*.
 
+**************
+Recording data
+**************
     Record calls
         Can be used to record external calls (see :ref:`call_recordings`).