Upgrade 18.03 to 19.1x (pelican-stretch)

English Installation/Configuration
1

Hello

I have problems for upgrading my wazo 18.03 to 19.12

  • Upgrade commands
root@pvoiall001:~# wazo-dist phoenix-stretch
root@pvoiall001:~# LANG=C
root@pvoiall001:~# wazo-upgrade
  • Multiple errors about service
Created symlink /etc/systemd/system/multi-user.target.wants/wazo-provd.service -> /lib/systemd/system/wazo-provd.service.
Job for wazo-provd.service failed because of unavailable resources or another system error.
See "systemctl status wazo-provd.service" and "journalctl -xe" for details.
wazo-provd.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-agid.service -> /lib/systemd/system/wazo-agid.service.
Job for wazo-agid.service failed because the control process exited with error code.
See "systemctl status wazo-agid.service" and "journalctl -xe" for details.
wazo-agid.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-dird.service -> /lib/systemd/system/wazo-dird.service.
Job for wazo-dird.service failed because the control process exited with error code.
See "systemctl status wazo-dird.service" and "journalctl -xe" for details.
wazo-dird.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-agentd.service -> /lib/systemd/system/wazo-agentd.service.
Job for wazo-agentd.service failed because the control process exited with error code.
See "systemctl status wazo-agentd.service" and "journalctl -xe" for details.
wazo-agentd.service couldn't start.
Setting up wazo-calld (19.12~20190822.163516.b4fb728.deb9) ...

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-calld.service -> /lib/systemd/system/wazo-calld.service.
Job for wazo-calld.service failed because the control process exited with error code.
See "systemctl status wazo-calld.service" and "journalctl -xe" for details.
wazo-calld.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-phoned.service -> /lib/systemd/system/wazo-phoned.service.
Job for wazo-phoned.service failed because the control process exited with error code.
See "systemctl status wazo-phoned.service" and "journalctl -xe" for details.
wazo-phoned.service couldn't start.
  • Python errors / missing file
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/00-run-xivo-check-db...
Checking database...
        NOK (current revision is 28443bfc4fb1)
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/000-wazo-update-keys...
401 Client Error: UNAUTHORIZED for url: https://localhost:9497/0.1/token
401 Client Error: UNAUTHORIZED for url: https://localhost:9497/0.1/token
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/001-update-database-until-entity-tenant.sh...
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 128, in <module>
    main()
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 120, in main
    config = _load_config()
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 24, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 30, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/003-load-wazo-user-from-file.py...
load-wazo-user-from-file: 002-create-tenants-from_entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/004-load-wazo-user-external-api-from-file.py...
load-wazo-user-external-api-from-file: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/005-load-wazo-user-admin-from-file.py...
load-wazo-user-admin-from-file: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/006-update-database-until-head.sh...
update-database-until-head: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/01-remove_unused_packages.sh...
  • SQL Error
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/02-run-xivo-update-config...
Updating Wazo config
Traceback (most recent call last):
  File "/usr/sbin/xivo-create-config", line 192, in <module>
    main()
  File "/usr/sbin/xivo-create-config", line 41, in main
    config = load_config()
  File "/usr/sbin/xivo-create-config", line 49, in load_config
    result.update(load_config_dhcp(session))
  File "/usr/sbin/xivo-create-config", line 63, in load_config_dhcp
    Dhcp.network_interfaces,
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2659, in first
    ret = list(self[0:1])
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
    return list(res)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2761, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column dhcp.network_interfaces does not exist
LINE 1: ... dhcp_pool_start, dhcp.pool_end AS dhcp_pool_end, dhcp.netwo...
                                                             ^
 [SQL: 'SELECT dhcp.active AS dhcp_active, dhcp.pool_start AS dhcp_pool_start, dhcp.pool_end AS dhcp_pool_end, dhcp.network_interfaces AS dhcp_network_interfaces \nFROM dhcp \n LIMIT %(param_1)s'] [parameters: {'param_1': 1}]
Generating configuration
ERROR: generation from template 'dhcp' failed because all variables could not be expanded:
subnet #XIVO_DHCP_NET4_SUBNET# netmask #XIVO_DHCP_NET4_NETMASK# { /tmp/tmp.L1qM3nY4i8/dhcp/etc/dhcp/dhcpd_subnet.conf.head: option subnet-mask #XIVO_DHCP_NET4_NETMASK#; /tmp/tmp.L1qM3nY4i8/dhcp/etc/dhcp/dhcpd_subnet.conf.head: option routers #XIVO_DHCP_NET4_IP#;
  • Missing file / python error
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/09_provd_add_phonebook_ip.sh...
Could not read config file /var/lib/wazo-auth-keys/wazo-provd-cli-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-provd-cli-key.yml'
Traceback (most recent call last):
  File "/usr/bin/wazo-provd-cli", line 3, in <module>
    from wazo_provd_cli import cli
  File "/usr/lib/python2.7/dist-packages/wazo_provd_cli/cli.py", line 89, in <module>
    auth_client = AuthClient(username=key_file['service_id'], password=key_file['service_key'], **_CONFIG['auth'])
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 58, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 38, in main
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 27, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 33, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 34, in <module>
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 24, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 30, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 134, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 116, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 81, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 31, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 38, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 251, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 233, in main
    _auto_create_config()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 124, in _auto_create_config
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 109, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 117, in _load_key_file
    'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/30_provd_default_passwords...
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 46, in <module>
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 36, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 42, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 141, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 123, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 70, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 35, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 42, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 121, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 103, in main
    migrate_new_status()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 69, in migrate_new_status
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 34, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 41, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 102, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 84, in main
    reconfigure_all_devices()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 56, in reconfigure_all_devices
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 30, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 37, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 110, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 92, in main
    remove_null_values()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 65, in remove_null_values
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 30, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 37, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 139, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 121, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 80, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 31, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 38, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 144, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 126, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 82, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 32, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 39, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
  • services status
root@pvoiall001:~# wazo-service status
Checking database...
        NOK (current revision is 01b4c79b3d47)
Checking services...
        running         dahdi
        failed          wazo-plugind
        running         wazo-webhookd
        running         xivo-sysconfd
        running         xivo-confgend
        running         wazo-confd
        running         wazo-auth
        stopped         wazo-call-logd
        running         xivo-dxtora
        running         wazo-provd
        failed          wazo-agid
        stopped         asterisk
        stopped         xivo-amid
        failed          wazo-agentd
        failed          wazo-dird
        failed          wazo-phoned
        failed          wazo-calld
        stopped         wazo-websocketd
        failed          wazo-chatd
  • Restart
root@pvoiall001:~# wazo-service restart
Closing port 5060.
Waiting for services to stop successfully...
Waiting for services to start successfully...
        starting dahdi ... OK
        starting wazo-plugind ... OK
        starting wazo-webhookd ... OK
        starting xivo-sysconfd ... OK
        starting xivo-confgend ... OK
        starting wazo-confd ... OK
        starting wazo-auth ... OK
        starting xivo-dxtora ... OK
        starting wazo-provd ... OK
        starting wazo-agid ...Job for wazo-agid.service failed because the control process exited with error code.
See "systemctl status wazo-agid.service" and "journalctl -xe" for details.
 NOK
Opening port 5060.
  • status wazo-agid
root@pvoiall001:~# systemctl status wazo-agid
* wazo-agid.service - wazo-agid server
   Loaded: loaded (/lib/systemd/system/wazo-agid.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2020-02-19 09:00:53 EST; 26s ago
  Process: 34435 ExecStart=/usr/bin/wazo-agid (code=exited, status=1/FAILURE)
  Process: 34432 ExecStartPre=/usr/bin/install -d -o wazo-agid -g wazo-agid /var/run/wazo-agid (code=exited, status=0/SUCCESS)

Feb 19 09:00:53 pvoiall001 wazo-agid[34435]:     main()
Feb 19 09:00:53 pvoiall001 wazo-agid[34435]:   File "/usr/lib/python2.7/dist-packages/wazo_agid/bin/agid.py", line 58, in main
Feb 19 09:00:53 pvoiall001 wazo-agid[34435]:     key_config = _load_key_file(ChainMap(cli_config, file_config, _DEFAULT_CONFIG))
Feb 19 09:00:53 pvoiall001 wazo-agid[34435]:   File "/usr/lib/python2.7/dist-packages/wazo_agid/bin/agid.py", line 115, in _load_key_file
Feb 19 09:00:53 pvoiall001 wazo-agid[34435]:     return {'auth': {'service_id': key_file['service_id'],
Feb 19 09:00:53 pvoiall001 wazo-agid[34435]: KeyError: 'service_id'
Feb 19 09:00:53 pvoiall001 systemd[1]: wazo-agid.service: Control process exited, code=exited status=1
Feb 19 09:00:53 pvoiall001 systemd[1]: Failed to start wazo-agid server.
Feb 19 09:00:53 pvoiall001 systemd[1]: wazo-agid.service: Unit entered failed state.
Feb 19 09:00:53 pvoiall001 systemd[1]: wazo-agid.service: Failed with result 'exit-code'.
2

These lines are causing you all other errors.
If you can revert a snapshot, go for it and before the upgrade fix the following command:

  • wazo-auth-cli token create

If you cannot revert a snapshot, then it will be more complicated …

3

Hi,

thanks for the reply, I have try to execute wazo-auth-cli token create

The result seems to be the same :

root@pvoiall001:~# wazo-dist phoenix-stretch
root@pvoiall001:~# LANG=C
root@pvoiall001:~# wazo-auth-cli token create
'backend'
root@pvoiall001:~# wazo-upgrade

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-provd.service -> /lib/systemd/system/wazo-provd.service.
Job for wazo-provd.service failed because of unavailable resources or another system error.
See "systemctl status wazo-provd.service" and "journalctl -xe" for details.
wazo-provd.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-agid.service -> /lib/systemd/system/wazo-agid.service.
Job for wazo-agid.service failed because the control process exited with error code.
See "systemctl status wazo-agid.service" and "journalctl -xe" for details.
wazo-agid.service couldn't start.

Created symlink /etc/systemd/system/multi-user.target.wants/wazo-dird.service -> /lib/systemd/system/wazo-dird.service.
Job for wazo-dird.service failed because the control process exited with error code.
See "systemctl status wazo-dird.service" and "journalctl -xe" for details.

Could not read config file /var/lib/wazo-auth-keys/wazo-chatd-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-chatd-key.yml'
Created symlink /etc/systemd/system/multi-user.target.wants/wazo-chatd.service -> /lib/systemd/system/wazo-chatd.service.
Setting up wazo-agentd (19.11~20190717.180315.730aaf3.deb9) ...
Adding system user wazo-agentd
Adding user wazo-agentd to group www-data
Created symlink /etc/systemd/system/multi-user.target.wants/wazo-agentd.service -> /lib/systemd/system/wazo-agentd.service.
Job for wazo-agentd.service failed because the control process exited with error code.
See "systemctl status wazo-agentd.service" and "journalctl -xe" for details.
wazo-agentd.service couldn't start.
Setting up wazo-calld (19.12~20190822.163516.b4fb728.deb9) ...
Created symlink /etc/systemd/system/multi-user.target.wants/wazo-calld.service -> /lib/systemd/system/wazo-calld.service.
Job for wazo-calld.service failed because the control process exited with error code.
See "systemctl status wazo-calld.service" and "journalctl -xe" for details.
wazo-calld.service couldn't start.
Setting up wazo-phoned (19.12~20190814.201749.827e37b.deb9) ...
Created symlink /etc/systemd/system/multi-user.target.wants/wazo-phoned.service -> /lib/systemd/system/wazo-phoned.service.
Job for wazo-phoned.service failed because the control process exited with error code.
See "systemctl status wazo-phoned.service" and "journalctl -xe" for details.
wazo-phoned.service couldn't start.

Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/00-run-xivo-check-db...
Checking database...
        NOK (current revision is 28443bfc4fb1)
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/000-wazo-update-keys...
401 Client Error: UNAUTHORIZED for url: https://localhost:9497/0.1/token
401 Client Error: UNAUTHORIZED for url: https://localhost:9497/0.1/token
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/001-update-database-until-entity-tenant.sh...
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 128, in <module>
    main()
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 120, in main
    config = _load_config()
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 24, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/pre-start.d/002-create-tenants-from-entities.py", line 30, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/003-load-wazo-user-from-file.py...
load-wazo-user-from-file: 002-create-tenants-from_entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/004-load-wazo-user-external-api-from-file.py...
load-wazo-user-external-api-from-file: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/005-load-wazo-user-admin-from-file.py...
load-wazo-user-admin-from-file: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/006-update-database-until-head.sh...
update-database-until-head: 002-create-tenants-from-entities.py should be executed first
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/01-remove_unused_packages.sh...

Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/02-run-xivo-update-config...
Updating Wazo config
Traceback (most recent call last):
  File "/usr/sbin/xivo-create-config", line 192, in <module>
    main()
  File "/usr/sbin/xivo-create-config", line 41, in main
    config = load_config()
  File "/usr/sbin/xivo-create-config", line 49, in load_config
    result.update(load_config_dhcp(session))
  File "/usr/sbin/xivo-create-config", line 63, in load_config_dhcp
    Dhcp.network_interfaces,
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2659, in first
    ret = list(self[0:1])
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
    return list(res)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2761, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column dhcp.network_interfaces does not exist
LINE 1: ... dhcp_pool_start, dhcp.pool_end AS dhcp_pool_end, dhcp.netwo...
                                                             ^
 [SQL: 'SELECT dhcp.active AS dhcp_active, dhcp.pool_start AS dhcp_pool_start, dhcp.pool_end AS dhcp_pool_end, dhcp.network_interfaces AS dhcp_network_interfaces \nFROM dhcp \n LIMIT %(param_1)s'] [parameters: {'param_1': 1}]
Generating configuration
ERROR: generation from template 'dhcp' failed because all variables could not be expanded:
subnet #XIVO_DHCP_NET4_SUBNET# netmask #XIVO_DHCP_NET4_NETMASK# { /tmp/tmp.fpTc2sMyAF/dhcp/etc/dhcp/dhcpd_subnet.conf.head: option subnet-mask #XIVO_DHCP_NET4_NETMASK#; /tmp/tmp.fpTc2sMyAF/dhcp/etc/dhcp/dhcpd_subnet.conf.head: option routers #XIVO_DHCP_NET4_IP#;

'mail' configuration OK, skipping.
'system' configuration OK, skipping.
DONE
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/55-fix-service-discovery...
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/fix_paths_rights.sh...
Executing upgrade script /usr/share/xivo-upgrade/pre-start.d/fix_twisted_reactor...
Enabling Wazo services...
Closing port 5060.
Waiting for services to stop successfully...
Waiting for services to start successfully...
        starting dahdi ... OK
        starting wazo-plugind ... OK
        starting wazo-webhookd ... OK
        starting xivo-sysconfd ... OK
        starting xivo-confgend ... OK
        starting wazo-confd ... OK
        starting wazo-auth ... OK
        starting xivo-dxtora ... OK
        starting wazo-provd ... OK
        starting wazo-agid ...Job for wazo-agid.service failed because the control process exited with error code.
See "systemctl status wazo-agid.service" and "journalctl -xe" for details.
 NOK
Opening port 5060.
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/09_provd_add_phonebook_ip.sh...
Could not read config file /var/lib/wazo-auth-keys/wazo-provd-cli-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-provd-cli-key.yml'
Traceback (most recent call last):
  File "/usr/bin/wazo-provd-cli", line 3, in <module>
    from wazo_provd_cli import cli
  File "/usr/lib/python2.7/dist-packages/wazo_provd_cli/cli.py", line 89, in <module>
    auth_client = AuthClient(username=key_file['service_id'], password=key_file['service_key'], **_CONFIG['auth'])
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 58, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 38, in main
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 27, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/18_upgrade_official_plugins.py", line 33, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 34, in <module>
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 24, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/20_reset_unassociated_devices_to_autoprov", line 30, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 134, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 116, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 81, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 31, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/25_dird_tenant_migration.py", line 38, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 251, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 233, in main
    _auto_create_config()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 124, in _auto_create_config
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 109, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/26-dird-auto-create-config.py", line 117, in _load_key_file
    'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/30_provd_default_passwords...
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 46, in <module>
    config = load_config()
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 36, in load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/50-autoprov-generate-config.py", line 42, in _load_key_file
    return {'auth': {'username': key_file['service_id'],
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 141, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 123, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 70, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 35, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/55_provd_device_tenant_migration.py", line 42, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 121, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 103, in main
    migrate_new_status()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 69, in migrate_new_status
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 34, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/56_provd_device_new_status_migration.py", line 41, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 102, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 84, in main
    reconfigure_all_devices()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 56, in reconfigure_all_devices
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 30, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/57-provd-reconfigure-all-devices.py", line 37, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 110, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 92, in main
    remove_null_values()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 65, in remove_null_values
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 30, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/60-provd-remove-null-values-config.py", line 37, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 139, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 121, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 80, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 31, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/61_webhookd_tenant_migration.py", line 38, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
Executing upgrade script /usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py...
Could not read config file /var/lib/wazo-auth-keys/wazo-upgrade-key.yml: [Errno 2] No such file or directory: '/var/lib/wazo-auth-keys/wazo-upgrade-key.yml'
Traceback (most recent call last):
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 144, in <module>
    main()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 126, in main
    migrate_tenants()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 82, in migrate_tenants
    config = _load_config()
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 32, in _load_config
    key_config = _load_key_file(ChainMap(file_config, _DEFAULT_CONFIG))
  File "/usr/share/xivo-upgrade/post-start.d/62_call_logd_tenant_migration.py", line 39, in _load_key_file
    'auth': {'username': key_file['service_id'], 'password': key_file['service_key']},
KeyError: 'service_id'
4

It is the problem, wazo-auth-cli cannot create a valid token.
You need to fix this command BEFORE running wazo-upgrade
The expected output should not be 'backend' but a token (ex: 6934b1be-db2d-4829-a099-93476837cd22)
You can check if your config file is OK

cat .config/wazo-auth-cli/050-credentials.yml

Can you check your previous upgrade if everything was OK?

less /var/log/xivo-upgrade.log
less /var/log/xivo-upgrade.log.1
zless /var/log/xivo-upgrade.log.2.gz
....
5

Hi,

Before launching the commands I have revert a snapshot just before the upgrade to 19.12

In cat .config/wazo-auth-cli/050-credentials.yml :

auth:
  username: wazo-auth-cli
  password: ****LAMp****AK70****
  backend: wazo_user

There is no previous upgrade, it’s a fresh install on 18.03 with a backup restoration from my production server (18.03)

I have try to execute wazo-auth-cli token createon my production server and I have the same result : ‘backend’

6

Oh yeah you’re right, there is a bug in 18.03 with wazo-auth-cli :sweat:
Can you try to use the password to create a token

curl -kX POST -u wazo-auth-cli:<your-password> --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"backend": "wazo_user", "expiration":10}' 'https://localhost:9497/0.1/token'

It’s a fresh install from the ISO or script?

7

Thanks, it works with the curl command but the command : wazo-auth-cli token create still not working

But I have try to execute wazo-upgrade :

root@pvoiall001:~# curl -kX POST -u wazo-auth-cli:pass --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"backend": "wazo_user", "expiration":3600}' 'https://localhost:9497/0.1/token'
root@pvoiall001:~# wazo-dist phoenix-stretch
root@pvoiall001:~# LANG=C
root@pvoiall001:~# wazo-upgrade

The upgrade was sucessful

Thanks a lot!

It’s a fresh install from the script with a debian 9 minimal

8

:anguished: … take a snapshot quick !!! :laughing:

It looks like a race condition or something that doesn’t happen systematically. It’s really hard to debug
Well if it happen on your future upgrade, the next step will be to send the complete logs

  • upgrade : /var/log/wazo-upgrade.log
  • wazo-auth : /var/log/wazo-auth.log

Usually everything should be fine. We test this kind of upgrade each day: https://jenkins.wazo.community/view/Upgrade/

9

Good morning all,
I found myself in this same situation, and did my research with an installation under a “VM”.
And after several hours, I found the reason for the problem !
The problem deprives the data construction of the backup, which does not keep the .config/wazo-auth-cli/050-credentials.yml.
This means that if you do a new installation, and you restore a backup, you will have on the one hand, a password restores in the database, and another password which comes from a new installation. .
What I did, therefore, was to make a backup in “PLAIN” mode of the asterisk database, then prepare an update request, which I used after, having restored my backup for version 18.03:
sudo -u postgres psql -d asterisk -c "UPDATE public.auth_user SET password_hash = '********************************************************************************************************************************'::text, password_salt = '\x********************************************************************************************************************************'::bytea WHERE username = 'wazo-auth-cli';"

In conclusion, we can talk about a bug, but which is just linked to this command under version 18.03 :
xivo-backup data /var/tmp/data-manual

To automatically solve this problem, you should create a script, which can regenerate a new password for the auth_user table, and the user wazo-auth-cli, as well as write it in the .config/wazo-auth-cli/050-credentials.yml.

If you want to solve this problem right away, while keeping in mind that I am sharing a password which should be private, here is the solution :
sudo -u postgres psql -d asterisk -c "UPDATE public.auth_user SET password_hash = ‘dad8bc990f0bd6d91d66d0c5b9e6de7b82d270cde9f0c7bb346d099d9c54292220c41b01c2fd1a9a09e1e17401039811e6862cb934d79b1bb4129494b56c0d75’::text, password_salt = ‘\x8b5b24fa13f1babb551bae1de65698e4021e5bb69d3537bdd1babfebbadf769c251685fb49eeb2a4a628bb8b4ffabb4162b7ec5b01a466ceeb6df7a280ac7aea’::bytea WHERE username = ‘wazo-auth-cli’;"

New password in .config/wazo-auth-cli/050-credentials.yml file
password: ADOPUPhZq9jAw4d9MsCQ

Modification test:
curl -kX POST -u wazo-auth-cli:ADOPUPhZq9jAw4d9MsCQ --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"backend": "wazo_user", "expiration":10}' 'https://localhost:9497/0.1/token'

I advise you to follow my first tips, which will allow you to have your own password.

Hope this will help many people !

10

How do you have generate salt and hash for the new password ?
If I understand, ADOPUPhZq9jAw4d9MsCQ is the new password and your register hash and salt in db … how i do it ?

11

Hello,
I paste the commands that will allow you to do a Recovery with the root user :

wazo-service stop
dpkg --purge --force-all wazo-auth
rm -r ~/.config/wazo-auth-cli
sudo -u postgres psql -d asterisk -c “DELETE FROM public.auth_user WHERE username = ‘wazo-auth-cli’;”
apt-get install wazo-auth

### Verification of writing in the database ###

sudo -u postgres psql -d asterisk -c “SELECT * FROM public.auth_user WHERE username = ‘wazo-auth-cli’;” -o “/tmp/auth_user_wazo-auth-cli.txt”

!!! I advise you to test this on a VM, or a few things that will allow you to restore everything, in case of malfunction. !!!

12

it seems to work fine. thks
after upgrade to 19.12, schedules are not working.

GET https://localhost:9486/1.1/schedules?recurse=False: [“Unexpected error: ‘NoneType’ object has no attribute ‘id’”]

is displayed in wazo-ui
Any idea ?

13

Hi, can you open another thread with your issue and the wazo-confd traceback? (/var/log/wazo-confd.log)