[SOLVED] Migrating srv-x64-ubuntu to srv-arm64-ubuntu

[b.ald]

WAPT Discovery -- 2.6.1.17472 -- SRV Ubuntu 24.04

Good morning,

I had set up a wapt server on the oracle cloud on a micro-x64 instance (1/8 ocpu / 1 GB RAM), everything works perfectly except that you have to be a little more patient given the performance of the server (limit of 50Mb/sec of upload bandwidth).
I wanted to migrate the server to an arm-x64 instance (better performance and 1Gb/sec/ocpu of bandwidth).

so I backed up

Code: Select all

/opt/wapt/conf
/opt/wapt/waptserver/ssl
/var/www/wapt*
/var/www/*.json

and dump the database.

I started the installation on the arm64 VM, but it seems there's an issue with the postconf script:

Code: Select all

root@xxxx:/opt/wapt/waptserver/ssl# /opt/wapt/waptserver/scripts/postconf.sh
/opt/wapt/waptserver/scripts/postconf.sh: line 12: 20784 Segmentation fault      (core dumped) NCURSES_NO_UTF8_ACS=1 PYTHONPATH=/opt/wapt /opt/wapt/bin/python -I /opt/wapt/waptserver/scripts/postconf.py $@

From what I've been able to find, it crashes with the new Python 3.10 environment when importing "setuphelper" into postconf.py. (But I'm probably wrong and I haven't investigated further.)



To get out of there, I purged everything and installed the latest 2.5 and there were no problems.
I migrated

Code: Select all

/opt/wapt/conf
/opt/wapt/waptserver/ssl
/var/www/wapt*
/var/www/*.json

but the database restoration fails (probably not the same schemas since I was on the latest 2.6)

So I have the server running on the latest 2.5, I have retrieved the server config and my packages and the host files but the console inventory remains empty (because there is no database restore?) and I cannot then upgrade to the latest 2.6 because of the error during postconf.

(To re-register a machine - agent 2.6 - I haven't found any other solution so far than to regenerate a waptagent 2.5 and do a "wapt-get install --force" to overwrite the installed agent 2.6, but I have to go to each machine to do this)

Commander.

[b.ald]

Good morning,

I'm back with a few more ideas, even if I don't have a solution (and I wouldn't find one anyway ^^)

So, the postconf problem which crashes in "segfault" does indeed occur from the first 2.6.1.
I tried launching postconf.py with each Python environment (well, I'm not a developer... I'll probably say some stupid things, but anyway)

Code: Select all

/opt/wapt/bin/python3.10
/opt/wapt/bin/python3.9
/opt/wapt/bin/python*oopsjesaisplusdsl*

but I got the same memory error when I tested it.

In short, I installed the latest version 2.6.0 (wapt-2.6.0.17392-0ac7bf48)

Code: Select all

cd /tmp

wget https://wapt.tranquil.it/wapt/releases/wapt-2.6.0.17392-0ac7bf48/tis-waptsetup-windows-2.6.0.17392-0ac7bf48-debian-10-amd64.deb
wget https://wapt.tranquil.it/wapt/releases/wapt-2.6.0.17392-0ac7bf48/tis-waptserver-2.6.0.17392-0ac7bf48-arm64.deb

apt install ./tis-waptserver-2.6.0.17392-0ac7bf48-arm64.deb ./tis-waptsetup-windows-2.6.0.17392-0ac7bf48-debian-10-amd64.deb

I then executed the

Code: Select all

/opt/wapt/waptserver/scritp/postconf.sh -s

who was able to get started.

However, there was already an issue in this version at the end of the postconf that I had already managed to work around/resolve before.
It was quite explicit:

Code: Select all

Output:Traceback (most recent call last):
  File "/opt/wapt/wapt-scanpackages.py", line 140, in <module>
    main()
  File "/opt/wapt/wapt-scanpackages.py", line 101, in main
    res2 = len(update_packages_table(conf, wapt_path))
  File "/opt/wapt/wapt-scanpackages.py", line 46, in update_packages_table
    load_db_config(conf, conf.get('db_schema'))
  File "/opt/wapt/waptserver/model.py", line 105, in load_db_config
    with wapt_db:
  File "/opt/wapt/lib/python3.9/site-packages/peewee.py", line 481, in inner
    return getattr(self.obj, method)(*args, **kwargs)
  File "/opt/wapt/lib/python3.9/site-packages/peewee.py", line 3231, in __enter__
    self.connect()
  File "/opt/wapt/lib/python3.9/site-packages/playhouse/pool.py", line 133, in connect
    ret = super(PooledDatabase, self).connect(reuse_if_open)
  File "/opt/wapt/lib/python3.9/site-packages/peewee.py", line 3263, in connect
    self._state.set_connection(self._connect())
  File "/opt/wapt/lib/python3.9/site-packages/playhouse/pool.py", line 83, in inner
    return fn(self, *args, **kwargs)
  File "/opt/wapt/lib/python3.9/site-packages/playhouse/pool.py", line 177, in _connect
    conn = super(PooledDatabase, self)._connect()
  File "/opt/wapt/lib/python3.9/site-packages/playhouse/postgres_ext.py", line 476, in _connect
    conn = super(PostgresqlExtDatabase, self)._connect()
  File "/opt/wapt/lib/python3.9/site-packages/peewee.py", line 3991, in _connect
    raise ImproperlyConfigured('Postgres driver not installed!')
peewee.ImproperlyConfigured: Postgres driver not installed!

From what I understand, the postgres driver was missing in the python3.9 environment (psycopg2), so I did the following:

So afterwards, after doing some research, it turned out to be "psycopg2-binary" and not "psycopg2"... (apparently to avoid possible "compilation dependency" problems...?)

Code: Select all

/opt/wapt/bin/python3.9 -m pip install psycopg2-binary

and I restarted the post-conference meeting which was able to finish.

This time I didn't wait and I redid the procedure to restore the database (which finished correctly, I had already restored everything else the first time).
I restarted a postconf (necessary? I think that's the first thing that postconf does, edit/update the database or... I don't really know).

and (after waptsetup -> install -> open console) upon restarting the console, a small relief:

251006_00h06m16s_1759701976.png (235.73 KB) Viewed 21392 times

However, apart from my admin position, all agents are on 2.6.1 (so n+1) but everything seems to be working (inventory / update / install / waptexit / etc)

I did regenerate an agent in version 2.6.0.17392, but then I wondered what the best course of action would be:

• Leave it as is?

• Overwrite agent 2.6.1 with version 2.6.0.17392 using a force install?

I'm going to stay on 2.6.0.17392, if I go back to 2.6.1, I don't think the postconf will work.

Commander.

[dcardon]

Hello Bastien,

I just tested it on a Debian 12 arm64 system and the server installation went smoothly. I don't have an Ubuntu 24.04 arm64 system handy to check.

What CPU do you have on your Amazon instance? The test I just ran was on a Mac mini using Parallels

Code: Select all

root@debian12:~# lscpu 
Architecture:             aarch64
  CPU op-mode(s):         64-bit
  Byte Order:             Little Endian
CPU(s):                   2
  On-line CPU(s) list:    0,1
Vendor ID:                ARM
  BIOS Vendor ID:         Apple
  Model name:             -
    BIOS Model name:      Apple Silicon None CPU @ 2.0GHz
    BIOS CPU family:      257
    Model:                0
    Thread(s) per core:   1
    Core(s) per cluster:  2
    Socket(s):            1
    Cluster(s):           1
    Stepping:             r0p0
    BogoMIPS:             48.00
    Flags:                fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp 
                          sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp flagm2 frint

Sincerely,

Denis

[dcardon]

Hello again Bastien,

I just did a test on a Debian 13 arm64 in virtual Parallels on the same Mac Mini, and it works.

However, on a Debian 13 arm64 VM on a vbox on a Win11 arm64 (Altra Ampere processor), I also have a segfault when importing setuphelpers.

But on the other hand, the problem isn't with psycopg2, it loads correctly (see below).

Capture.PNG (5.74 KB) Viewed 21345 times

I'll dig tomorrow.

Sincerely,

Denis

[dcardon]

Hello again,

Actually, it's the loading of the waptlicences.so file (which is misnamed because it's actually "waptfpclibs.so", i.e., the Wapt libraries in Lazarus/FPC exposed to Python). Now it's directly included in wapt-get.bin and should no longer be loaded separately under Linux, but the postconf script was (this has just been fixed in the master git repository) calling /opt/wapt/bin/python instead of /opt/wapt/wapt-get.bin (which is also the Python interpreter).

For your information, we switched to /opt/wapt/wapt-get.bin to better isolate the wapt process and its python scripts from the system environment: the cpython project loader, i.e. python.exe (or the equivalent under Linux) is far too permissive with regard to what it loads.

Therefore, you simply need to modify the following line in the file /opt/wapt/waptserver/scripts/postconf.sh

Code: Select all

NCURSES_NO_UTF8_ACS=1 PYTHONPATH=/opt/wapt /opt/wapt/wapt-get.bin -I /opt/wapt/waptserver/scripts/postconf.py $@

And the post-conference should be working. Thanks for bringing it to our attention; we should have fixed that sooner.

Sincerely,

Denis

[b.ald]

Hi Denis,

First of all, thank you very much.

The instance is running on the Oracle cloud. So I wasn't really surprised (even relieved) to read that it was crashing under vBox.
It's an A1-arm instance, so it's an Ampere processor... I would have preferred to go with Debian, but Oracle's free tier only offers Ubuntu (no Debian) and you can't import your own image.

As for "psycopg2", the problem occurred when I manually installed version 2.6.0.17392, specifically when using "wget". I should have been clearer and gone into a bit more detail... (by the way, there's no "segfault" in this version.)

And yes, I remember reading somewhere (but I can't remember where) that you found the Python project binary too permissive (but I'm not a developer by trade, so I'm still wondering why ^^) and that wapt-get.bin was actually the Python binary that you redeveloped, if my memory serves me right.
Thanks for the detailed information; I'll have to "decipher" it a bit ^^

So, the switch to /opt/wapt/wapt-get.bin is from version 2.6.1 onwards?

I'm going to update the server and modify postconf.sh.
Thanks again.

Best regards,
Bastien

[b.ald]

Hello again,

The postconf starts and finishes correctly (or at least I think so). Thank you very much.

Just :

Code: Select all

Waptsetup agent installer:
    Version available on server: (True, '2.6.1.17472')
    You can download "waptsetup" Windows agent installer at URL: @@@@@
    Check the Code signing signature and the hash of the installer: Sha256: 718ad50ee9ad965206ced382260c7b882b9bf2d26265c6db37e955fd5baab0c8


Traceback (most recent call last):
  File "<string>", line 1404, in <module>
  File "<string>", line 1315, in main
  File "/opt/wapt/waptutils.py", line 2263, in run
    raise CalledProcessErrorOutput(proc.returncode, cmd, ''.join(output))
waptutils.CalledProcessErrorOutput: Command b'/opt/wapt/wapt-scanpackages.sh -b /var/www/wapt' returned non-zero exit status 139.
Output:/opt/wapt/wapt-scanpackages.sh: line 30: 72915 Segmentation fault      sudo -u wapt CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1 PYTHONNOUSERSITE=1 PYTHONPATH="${WAPT_HOME}" "${WAPT_HOME}"/bin/python -I "${WAPT_HOME}"/wapt-scanpackages.py $@

CalledProcessErrorOutput: Command b'/opt/wapt/wapt-scanpackages.sh -b /var/www/wapt' returned non-zero exit status 139.
Output:/opt/wapt/wapt-scanpackages.sh: line 30: 72915 Segmentation fault      sudo -u wapt CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1 PYTHONNOUSERSITE=1 PYTHONPATH="${WAPT_HOME}" "${WAPT_HOME}"/bin/python -I "${WAPT_HOME}"/wapt-scanpackages.py $@

I'm going to modify wapt-scanpackages.sh in the same way and replace "/bin/python" (well... I suppose so?)

Regards,
Bastien




[edit]: I ended up not making the change to "wapt-scanpackage" because the "wapt-get" command works:

Code: Select all

root@my-wapt:~# wapt-get scan-packages /var/www/wapt/
Using config file: /opt/wapt/wapt-get.ini
Packages filename : /var/www/wapt/Packages
Processed packages :
  /var/www/wapt/zth-testscanpkg_0-1_PROD.wapt
Skipped packages :
  /var/www/wapt/zth-ms-store-winget_2025.717.1857.0-78_windows_10.0_PROD.wapt
  /var/www/wapt/zth-rustdesk_1.4.2-3_x64_windows_10_PROD_7c3f13b8f4e027b6494b2a1dc7e20407.wapt

+ everything is functional in the console and I'm not sure when this script can be called.
Thanks anyway, the server and agents are indeed on 2.6.1 and everything seems to be working. (Perhaps /bin/python will need to be replaced in other places)

[dcardon]

Hi Bastien,

yes, the /opt/wapt/wapt-scanpackages.sh file has the same problem as the postconf file. We'll modify that as well.

Regarding the switch to the Python loader wapt-get.exe/wapt-get.bin (which is written in Lazarus/FPC), we implemented that in version 2.6.0. If you'd like to better understand why, you can run a process explorer on a Windows Python startup. Beyond the PYTHONPATH issue (which is already quite problematic), there's also the problem of loading DLLs/SO files, which can be retrieved from unexpected locations. Not to mention libraries that look for paths on Windows as if they were Linux systems (like searching for default files in c:\etc\...).

To add to that, the Python engine itself is integrated into the python310.dll/libpython310.so file, and there are no changes there, only to the loader.

The integration of the waptlicences.dll/waptlicences.so library into the wapt-get binary must have happened at the same time. This was also due to problems with the dynamic linker, which sometimes gets confused.

In short, dynamic languages ​​like Python are incredibly useful, but you really have to be careful when using them as a sub-component of a product.

Sincerely,

Denis

[b.ald]

Hi Denis,

Thanks for the clarification, but I confess I don't understand everything. I don't know what a dynamic linker is, for example, but I'll keep that in mind and look into it.
I did a "quick" career change into IT. I took a few network/systems courses, but very few development courses (well, if you can call them that: conceptual data models/logical data models, Java class diagrams, etc.). I
've done a bit of scripting in Bash, but otherwise, my Python experience is limited to WAPT, as I'm practically the only one editing the packages and updating the servers. But I've never really used Python outside the WAPT environment via packages or the wapt-get shell.
I do a lot of "ctrl-clicking" in PyScripter and try to see and understand what's already in the setuphelpers/common folders or how to use this or that library (DLLs? and so on?). I think I used BeautifulSoup (a function?) in a package. But I'm not even sure I'm using the right terms, let alone if the Python lines I'm writing are "clean" ^^.

And I saw a case (I don't remember the details) where sites were loading JS DLLs from a reputable domain name that was later bought by Chinese companies who then injected malicious code into it.

For WAPT, is there a section or place where we can ask for advice/opinions regarding server and agent configuration (depending on their environment/context), security, etc., or not really?
I've mostly seen posts about "troubleshooting/support" or "bug reports/feedback."

It's mainly for the Discovery version where I'm not entirely sure of the best way to configure/secure it, especially if the server is directly exposed to the public (on Oracle's cloud in my case for the Discovery server).

Otherwise, it hasn't been long, that's for sure, but since the modification and update, I haven't encountered any bugs or errors via the console. I haven't made any other changes besides the postconf.sh file.
I can finally get feedback in the console because with the other, sluggish server, everything worked perfectly; downloads were admittedly very slow, but installs/updates/uninstalls all completed correctly.
But in the console, I only got a before/after view. ^^

Thanks again.

Best regards,
Bastien

[b.ald]

Hello,

I think this topic can be closed if you wish.

Regards,
Bastien