WAPT 1.3 to 1.5 Migration Error

[tech62]

Good morning,

After following the instructions on the wiki, I still encounter the error:
Version: .5.1.9-tisdeb9-4420-1134da77

Code: Select all

mongodb process running, need to migrate
sh: 1: mongoexport: not found
Traceback (most recent call last):
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 209, in <module>
    upgrade2postgres()
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 151, in upgrade2postgres
    create_import_data(ip='127.0.0.1', fn=data_import_filename)
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 72, in create_import_data
    d = json.load(os.popen('mongoexport -h %s -d wapt -c hosts  --jsonArray' % ip))
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Traceback (most recent call last):
  File "/opt/wapt/waptserver/scripts/postconf.py", line 511, in <module>
    main()
  File "/opt/wapt/waptserver/scripts/postconf.py", line 338, in main
    mongo_update_status = check_mongo2pgsql_upgrade_needed(waptserver_ini)
  File "/opt/wapt/waptserver/scripts/postconf.py", line 241, in check_mongo2pgsql_upgrade_needed
    run_verbose("sudo -u wapt /usr/bin/python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres")
  File "/opt/wapt/waptserver/scripts/postconf.py", line 68, in run_verbose
    output = subprocess.check_output(*args, shell=True, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'sudo -u wapt /usr/bin/python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres' returned non-zero exit status 1

Any idea what the problem might be?
Machine running Debian 9.*
THANKS !

[tech62]

After running `apt-get install mongo-tools`, I get this:

Code: Select all

could not change directory to "/root": Permission non accordée
2017-12-08T14:49:47.069+0100    connected to: 127.0.0.1
2017-12-08T14:49:47.138+0100    exported 6 records
WARNING:root:Traceback (most recent call last):
  File "/opt/wapt/waptserver/waptserver_model.py", line 424, in update_host_data
    existing = Hosts.select(Hosts.uuid, Hosts.computer_fqdn).where(Hosts.uuid == uuid).first()
  File "/opt/wapt/lib/site-packages/peewee.py", line 4941, in select
    query = SelectQuery(cls, *selection)
  File "/opt/wapt/lib/site-packages/peewee.py", line 3016, in __init__
    self.require_commit = self.database.commit_select
  File "/opt/wapt/lib/site-packages/peewee.py", line 439, in __getattr__
    raise AttributeError('Cannot use uninitialized Proxy.')
AttributeError: Cannot use uninitialized Proxy.

CRITICAL:root:Error updating data for [SERVERID] : AttributeError: Cannot use uninitialized Proxy.
Traceback (most recent call last):
  File "/opt/wapt/waptserver/scripts/postconf.py", line 511, in <module>
    main()
  File "/opt/wapt/waptserver/scripts/postconf.py", line 338, in main
    mongo_update_status = check_mongo2pgsql_upgrade_needed(waptserver_ini)
  File "/opt/wapt/waptserver/scripts/postconf.py", line 241, in check_mongo2pgsql_upgrade_needed
    run_verbose("sudo -u wapt /usr/bin/python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres")
  File "/opt/wapt/waptserver/scripts/postconf.py", line 68, in run_verbose
    output = subprocess.check_output(*args, shell=True, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'sudo -u wapt /usr/bin/python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres' returned non-zero exit status 1

[tech62]

By running: python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres

I get:

Code: Select all

upgrading data from mongodb to postgresql
psql: FATAL:  le rôle « root » n'existe pas
Traceback (most recent call last):
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 209, in <module>
    upgrade2postgres()
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 144, in upgrade2postgres
    val = subprocess.check_output("""  psql wapt -c " SELECT datname FROM pg_database WHERE datname='wapt';   " """, shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '  psql wapt -c " SELECT datname FROM pg_database WHERE datname='wapt';   " ' returned non-zero exit status 2

[tech62]

Postgres log:

Code: Select all

déc. 08 15:10:27 wapt systemd[1]: Starting PostgreSQL RDBMS...
déc. 08 15:10:27 wapt systemd[1]: Started PostgreSQL RDBMS.
déc. 08 15:10:27 wapt systemd[1]: postgresql.service: Failed to reset devices.list: Operation not permitted
déc. 08 15:13:41 wapt systemd[1]: postgresql.service: Failed to reset devices.list: Operation not permitted
déc. 08 15:15:36 wapt systemd[1]: postgresql.service: Failed to reset devices.list: Operation not permitted
déc. 08 15:15:37 wapt systemd[1]: postgresql.service: Failed to reset devices.list: Operation not permitted
déc. 08 15:26:01 wapt systemd[1]: postgresql.service: Failed to reset devices.list: Operation not permitted

[sfonteneau]

Code: Select all

sudo -u wapt python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres

[tech62]

Here is the result of the command:

Code: Select all

sudo -u wapt python /opt/wapt/waptserver/waptserver_upgrade.py upgrade2postgres

Code: Select all

Upgrading from mongodb to postgres
[uwsgi]
http-socket = 127.0.0.1:8080
master = true
processes = 16
wsgi=waptserver:app
chdir=/opt/wapt/waptserver/
max-requests=100
uid=wapt
gid=www-data
enable-threads = true

[options]
#################################mongodb_port=27017
#################################mongodb_ip=127.0.0.1

#wapt_folder=/var/www/wapt/
#waptwua_folder=/var/www/waptwua
#waptserver_port=8080
wapt_user=[SECRET]
wapt_password=[SECRET]
wapt_folder = /var/www/wapt
server_uuid = [SECRET]
#loglevel=warning

#client_tasks_timeout=0.3
#client_connect_timeout=1
#client_read_timeout=10

upgrading data from mongodb to postgresql
could not change directory to "/root": Permission non accordée

dumping mongodb data in /tmp/waptupgrade_20171208-Dec:26:1512750374.json
Read mongo data from 127.0.0.1...
2017-12-08T17:26:14.064+0100    connected to: 127.0.0.1
2017-12-08T17:26:14.131+0100    exported 6 records
6 records read.
File /tmp/waptupgrade_20171208-Dec:26:1512750374.json done.
Loading /tmp/waptupgrade_20171208-Dec:26:1512750374.json
6 recs to load
WARNING:root:Traceback (most recent call last):
  File "/opt/wapt/waptserver/waptserver_model.py", line 424, in update_host_data
    existing = Hosts.select(Hosts.uuid, Hosts.computer_fqdn).where(Hosts.uuid == uuid).first()
  File "/opt/wapt/lib/site-packages/peewee.py", line 4941, in select
    query = SelectQuery(cls, *selection)
  File "/opt/wapt/lib/site-packages/peewee.py", line 3016, in __init__
    self.require_commit = self.database.commit_select
  File "/opt/wapt/lib/site-packages/peewee.py", line 439, in __getattr__
    raise AttributeError('Cannot use uninitialized Proxy.')
AttributeError: Cannot use uninitialized Proxy.

CRITICAL:root:Error updating data for [SECRET] : AttributeError: Cannot use uninitialized Proxy.
Error for [SECRET] : AttributeError: Cannot use uninitialized Proxy.
Traceback (most recent call last):
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 153, in upgrade2postgres
    load_json(filenames=data_import_filename)
  File "/opt/wapt/waptserver/waptserver_upgrade.py", line 106, in load_json
    wapt_db.rollback()
  File "/opt/wapt/lib/site-packages/peewee.py", line 439, in __getattr__
    raise AttributeError('Cannot use uninitialized Proxy.')
AttributeError: Cannot use uninitialized Proxy.
Exception while loading data, please check current configuration

[tech62]

UP

[htouvet]

A database initialization is missing from the waptserver_upgrade code.

2 lines to add:
Add

Code: Select all

from waptserver_model import load_db_config

in the imports at the top of the code

Add

Code: Select all

    conf = waptserver_config.load_config(options.configfile)

After

Code: Select all

    (options, args) = parser.parse_args()

Code: Select all

--- a/waptserver/waptserver_upgrade.py
+++ b/waptserver/waptserver_upgrade.py
@@ -48,6 +48,7 @@ sys.path.insert(0, os.path.join(wapt_root_dir, 'lib'))
 sys.path.insert(0, os.path.join(wapt_root_dir, 'lib', 'site-packages'))
 
 from waptserver_config import __version__
+from waptserver_model import load_db_config
 
 import logging
 import ConfigParser
@@ -196,6 +197,7 @@ if __name__ == '__main__':
         help='test prefix for fqdn and uuid for load testing (for development only)')
 
     (options, args) = parser.parse_args()
+    conf = waptserver_config.load_config(options.configfile)
 
     utils_set_devel_mode(options.devel)
     if options.loglevel is not None: 

Night build: https://wapt.tranquil.it/wapt/nightly/w...-ef7b7b4b/

[tech62]

Thanks for the feedback!
When will the change be implemented in the warehouse?

[htouvet]

I have updated the RC 1.5 repository