Page 1 of 1

[RESOLVED] CPU at 99%

Published: October 23, 2019 - 10:08 AM
by erickeke
Hello,

Since updating wapt + debian following the wiki procedure, my python process is always at 99.9%:

771 wapt 20 0 198608 83228 13776 R 99,9 4,1 5:20.27 python

I restart the service, it works fine for 30s then it goes into a loop.

By running `strace -f -p pid`, we see that at the moment the CPU goes haywire, there's an avalanche of calls in an infinite loop:
(3, [],
0
1023
epoll_wait
)
`
=
0`
,1023, 0) = 0
epoll_wait(3, [], 1023, 0) = 0
epoll_wait(3, [], 1023, 0) = 0
epoll_wait(3, [], 1023, 0) = 0

I am running on an ESX, I tried changing the type of network card: switching to e1000 but it didn't change anything.

Best regards

Versions:

Debian 9.11
Kernel: 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20) x86_64 GNU/Linux

WAPT Server version: 1.7.4
WAPT Agent version: 1.7.4.6223
WAPT Setup version: 1.7.4.6223
WAPT Deploy version: 1.7.4.6223
Database status: OK (1.7.4.0)

Re: CPU at 99%

Published: October 23, 2019 - 11:38 AM
by sfonteneau
Hello,

we have just identified the problem.

We have upgraded some libraries to the latest version. However, one of the libraries is causing an issue.

A fix for the library has just been released:

https://github.com/miguelgrinberg/pytho ... CHANGES.md

Prevent heavy CPU usage when decoding payloads:
https://github.com/miguelgrinberg/pytho ... 8454f5e50e

We will be releasing a new version of wapt today.

Simon

Re: CPU at 99%

Published: October 23, 2019 - 12:26
by erickeke
How effective is it? :-)

I tested it by just retrieving the corrected payload.py (and deleting payload.pyc), but I didn't see any difference. I'm probably doing it wrong.

Regards

Re: CPU at 99%

Published: October 23, 2019 - 1:33 PM
by sfonteneau
Can you try this version?

https://wapt.tranquil.it/wapt/nightly/w ... -b30b1b1a/

No need to upgrade the agents

Re: CPU at 99%

Published: October 23, 2019 - 4:57 PM
by htouvet
There is a fix in the wapt-1.7.4.6229 release.

https://wapt.tranquil.it/wapt/releases/wapt-1.7.4.6229/

Re: CPU at 99%

Published: October 24, 2019 - 10:08 AM
by erickeke
Sorry, I wasn't in the office to test the nightly build yesterday afternoon.

I installed version 6229 and indeed the problem disappeared! :-)

Thank you so much, you guys are awesome!