[cmake-developers] kernel log segfault warning messages when building/testing CMake

[Alan W. Irwin]

My "merlin" computer automatically builds and tests (including the
PLplot contract test) CMake each day at 4:32 Pacific time, and I have
noticed for some time now that those successful "merlin" tests (clean for the
CMake dashboard, but with 8 compile warnings but no errors for the
KWSys dashboard that is also generated by the CMake test) are always
accompanied by segfault warnings in the kernel log file for my Debian
Buster system. At least some of those segfaults seem to
be associated with the KWSys part of this test. Here are those log
warnings for today (taken from /var/log/messages):

======================
Oct 23 04:32:47 merlin kernel: [145023.933476] cmsysTestProces[5466]: segfault at 4 ip 000055d23f362e3b sp 00007ffda48d1940
error 6 in cmsysTestProcess[55d23f362000+6000]
Oct 23 04:32:47 merlin kernel: [145023.933483] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 f
f ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00

Oct 23 04:32:47 merlin kernel: [145023.935919] cmsysTestProces[5472]: segfault at 4 ip 000055b7f515be3b sp 00007ffd46054c90
error 6 in cmsysTestProcess[55b7f515b000+6000]
Oct 23 04:32:47 merlin kernel: [145023.935928] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 f
f ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00

Oct 23 04:33:48 merlin kernel: [145084.853316] Crash[6355]: segfault at 0 ip 000055f36cfd8135 sp 00007ffcc0d2d0f0 error 6 i
n Crash[55f36cfd8000+1000]
Oct 23 04:33:48 merlin kernel: [145084.853321] Code: 2f 00 00 01 5d c3 0f 1f 80 00 00 00 00 c3 0f 1f 80 00 00 00 00 e9 7b f
f ff ff 55 48 89 e5 48 c7 45 f8 00 00 00 00 48 8b 45 f8 <c7> 00 01 00 00 00 b8 00 00 00 00 5d c3 66 2e 0f 1f 84 00 00 00 00

Oct 23 04:34:33 merlin kernel: [145130.287867] kwsysTestProces[2964]: segfault at 4 ip 0000557e9dadce3b sp 00007fff85d3cce0 error 6 in kwsysTestProcess[557e9dadc000+6000]
Oct 23 04:34:33 merlin kernel: [145130.287875] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 ff ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00
Oct 23 04:34:35 merlin kernel: [145131.727495] kwsysTestProces[4382]: segfault at 4 ip 000055ccd8790e3b sp 00007ffc422fea20 error 6 in kwsysTestProcess[55ccd8790000+6000]
Oct 23 04:34:35 merlin kernel: [145131.727499] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 ff ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00
Oct 23 04:35:12 merlin kernel: [145169.367275] kwsysTestProces[4838]: segfault at 4 ip 0000563d98a43e3b sp 00007ffce04a4ab0 error 6 in kwsysTestProcess[563d98a43000+6000]
Oct 23 04:35:12 merlin kernel: [145169.367279] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 ff ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00
Oct 23 04:41:17 merlin kernel: [145533.853387] kwsysTestProces[17736]: segfault at 4 ip 000056237dea3e3b sp 00007ffc5cd97cf0 error 6 in kwsysTestProcess[56237dea3000+6000]
Oct 23 04:41:17 merlin kernel: [145533.853394] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 ff ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00
Oct 23 04:41:17 merlin kernel: [145533.855467] kwsysTestProces[17742]: segfault at 4 ip 000055ae9d89ce3b sp 00007fff23ff8b30 error 6 in kwsysTestProcess[55ae9d89c000+6000]
Oct 23 04:41:17 merlin kernel: [145533.855473] Code: 00 48 8d 3d 6f 56 00 00 e8 12 f5 ff ff 48 8b 3d db 83 00 00 e8 76 f4 ff ff 48 8b 3d ef 83 00 00 e8 6a f4 ff ff ba 2e 00 00 00 <c7> 04 25 04 00 00 00 00 00 00 00 48 8b 0d b3 83 00 00 be 01 00 00

======================

I don't notice any other platform on KWSys dashboard that also has the
8 build warnings, but it is likely my fairly recent gcc compiler
(Debian Buster ccache and gcc/g++ 8.2.0) is more sensitive to
problematic code in KWSys than prior versions of the gcc compiler
suite.

Anyhow, I suggest cleaning up the 8 build warnings for the KWSys
code to see if those are the source of some/all of these
segfaults. And if no CMake developer here has access to Debian Buster, I
would be glad to run preliminary tests of any of your proposed KWSys code
changes to make sure the compile warnings are gone.

I have attached a compressed version of the my_dashboard.cmake script
that created a while ago (with lots of advice from Brad) to configure
this CMake test in case there is anything problematic in there. For
example, I notice

set(ENV{CXXFLAGS} "-O3 -DNDEBUG")
set(ENV{CFLAGS} "-O3 -DNDEBUG")

in there in case that makes a difference with regard to the above KWSys compile warnings
that seem to appear just for my platform.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Rolf Eike Beer]

Post by Alan W. Irwin
My "merlin" computer automatically builds and tests (including the
PLplot contract test) CMake each day at 4:32 Pacific time, and I have
noticed for some time now that those successful "merlin" tests (clean for
the CMake dashboard, but with 8 compile warnings but no errors for the
KWSys dashboard that is also generated by the CMake test) are always
accompanied by segfault warnings in the kernel log file for my Debian
Buster system. At least some of those segfaults seem to
be associated with the KWSys part of this test. Here are those log

I get those daily, here from my sparc:

[1540252.631482] kwsysTestProces[44830]: segfault at 4 ip 0000000000011c68
(rpc 0000000000011c28) sp 00000000ffd7b670 error 1 in
kwsysTestProcess[10000+e000]
[1540255.178055] kwsysTestProces[45441]: segfault at 4 ip 0000000000011c68
(rpc 0000000000011c28) sp 00000000ff933670 error 1 in
kwsysTestProcess[10000+e000]
[1541099.090339] kwsysTestProces[65043]: segfault at 4 ip 0000000000011c68
(rpc 0000000000011c28) sp 00000000ffad96b0 error 1 in
kwsysTestProcess[10000+e000]
[1541099.096219] kwsysTestProces[65049]: segfault at 4 ip 0000000000011c68
(rpc 0000000000011c28) sp 00000000ff95f6b0 error 1 in
kwsysTestProcess[10000+e000]

This is intentional, there are test binaries that segfault to test if that is
properly detected by the signal handler/debug/ctest/whatever code.

Post by Alan W. Irwin
I don't notice any other platform on KWSys dashboard that also has the
8 build warnings, but it is likely my fairly recent gcc compiler
(Debian Buster ccache and gcc/g++ 8.2.0) is more sensitive to
problematic code in KWSys than prior versions of the gcc compiler
suite.

I run gcc 8.2 on castor, but I don't know why this does not trigger these
warnings.

Eike

[Alan W. Irwin]

On 2018-10-23 21:20+0200 Rolf Eike Beer wrote:

[...]

[These segfaults are] intentional, there are test binaries that segfault to test if that is
properly detected by the signal handler/debug/ctest/whatever code.

OK, and thanks for that info.

I have therefore changed the subject line appropriately.

Post by Alan W. Irwin
I don't notice any other platform on KWSys dashboard that also has the
8 build warnings, but it is likely my fairly recent gcc compiler
(Debian Buster ccache and gcc/g++ 8.2.0) is more sensitive to
problematic code in KWSys than prior versions of the gcc compiler
suite.

I run gcc 8.2 on castor, but I don't know why this does not trigger these
warnings.

It's possible it is the gcc/g++ flags I am using, but in any case
someone knowledgeable should look at those build warnings which
repeatedly show up on the "merlin" report for the the KWSys dashboard
to see if there is anything obvious that can be done to deal with the
source code issues that gcc/g++ has discovered for my Debian Buster
platform.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Alan W. Irwin]

The promised attachment concerning how my CMake test is configured.


__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

[Brad King]

Post by Alan W. Irwin
look at those build warnings which
repeatedly show up on the "merlin" report for the the KWSys dashboard

The warnings come from code generated by CMake. You're driving the
builds with CMake 3.7 which still generated code on which gcc 8 warns.
CMake 3.11 has the fix. If you drive with that or higher instead then
the warnings should go away.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Alan W. Irwin]

Post by Brad King

Post by Alan W. Irwin
look at those build warnings which
repeatedly show up on the "merlin" report for the the KWSys dashboard

The warnings come from code generated by CMake. You're driving the
builds with CMake 3.7 which still generated code on which gcc 8 warns.
CMake 3.11 has the fix. If you drive with that or higher instead then
the warnings should go away.

OK. The issue was I keep ctest-3.7.2 (and cmake-3.7.2) first on my PATH because I
want to make sure to test those versions for the PLplot case (since that is the
PLplot minimum version of CMake for now although I plan within a year or so to
bump that to 3.12.3 or higher once most Linux distributions provide that version).

To fix this issue, I left my PATH the same but I now use the explicit
system version of ctest (3.12.3 on Debian Buster) to launch the
script, and the script automatically figures out inside to use the
compatible version of cmake to launch other parts of the test rather
than cmake-3.7.2 from my PATH). And as you predicted the warnings go
away (see the most recent merlin results on the dashboard). So thanks
very much for your key help in getting this issue figured out.

I seem to have been the only one currently who has tried to use a
ctest version less than 3.11 to drive testing, but nevertheless, you
may still want to document that the minimum ctest version to drive
testing should be 3.11 to avoid others running into this issue in the
future.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers