[cmake-developers] A dashboard that reveals a ctest issue on Windows when there are no tests

[Alan W. Irwin]

Three dashboards for PLplot submitted today at <https://my.cdash.org/index.php?project=PLplot_git>
had a build warning concerning ctest. For example, <https://my.cdash.org/buildSummary.php?buildid=1574104> contained

Build Warnings (1)

*** WARNING non-zero return value in ctest from: C:\cmake-3.13.1-win64-x64\bin\cmake.exe

which I have never seen before, and the test summary section
contained (just) the following message

0 passed, 0 failed, 0 failed for timing, 0 not run.

(which I have also never seen before). A PLplot peculiarity is that
all tests depend on bash scripts so if bash.exe is not available (as
in this user's case) then there will be no ctest tests configured.

In an attempt to replicate this issue, I
forced bash not to be found on my own Linux box which guaranteed no
add_test logic was executed. However, instead of the above results
here was the reasonable PLplot "no-test" result from ctest (version 3.13.1):

***@merlin> ctest
Test project /home/software/plplot/HEAD/build_dir
No tests were found!!!
***@merlin> echo $?
0

So I would have expected the same reasonable "no-test" result on this
user's Windows machine (where he was also using ctest-3.13.1), but
instead he is generating the above peculiar results.

Can someone here shed some light on this issue? For example, could
this be a ctest bug for the "no-test" case that this user's platform
exposes? Or could this be a cdash issue when a dashboard is submitted
with the above reasonable "no-test" results I got on Linux?

Note, there cannot be much wrong with the normal PLplot configuration
of ctest when most of our ~30 tests are configured since I have
recently submitted (see, e.g.,
<https://my.cdash.org/index.php?project=PLplot_git&date=2018-12-02>)
successful dashboards for a fully loaded Linux systems, and Arjen
Markus has done the same thing previously for his fully loaded Cygwin
and MinGW-w64/MSYS2 platforms.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Brad King via cmake-developers]

Post by Alan W. Irwin
Build Warnings (1)
*** WARNING non-zero return value in ctest from: C:\cmake-3.13.1-win64-x64\bin\cmake.exe

That's in the "Build" section and indicates that the build command
exited with non-zero status. If you want to try to reproduce that
by hand, run the `cmake --build . --config Debug` command rather
than `ctest`.

`cmake --build`'s exit code just forwards from the native build tool.
It may be that MSBuild has chosen to exist with non-zero for some reason.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Alan W. Irwin]

Post by Brad King via cmake-developers

Post by Alan W. Irwin
Build Warnings (1)
*** WARNING non-zero return value in ctest from: C:\cmake-3.13.1-win64-x64\bin\cmake.exe

That's in the "Build" section and indicates that the build command
exited with non-zero status. If you want to try to reproduce that
by hand, run the `cmake --build . --config Debug` command rather
than `ctest`.
`cmake --build`'s exit code just forwards from the native build tool.
It may be that MSBuild has chosen to exist with non-zero for some reason.

Hi Brad:

Did you mean "ctest --build-and-test ..." rather than "cmake --build
..."? I assume if the build part of "ctest --build-and-test ..."
returned with some non-zero return code, then you would get a message
like the above that mentions ctest in a way that implies it is running
the show. But I cannot see any way you could get such a WARNING
message from "cmake --build ...". I hasten to add that I have been
completely content over the years to use cmake, make, and ctest
separately, and I have no experience using them in an
integrated way (e.g., "cmake --build ..." or "ctest --build-and-test
..."). So my comment is based on my quick reading the appropriate
sections of the man pages for cmake and ctest this morning, and I may
have missed some way that a cmake command could generate what appears
to be a message from ctest.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Alan W. Irwin]

Post by Alan W. Irwin

Post by Brad King via cmake-developers

Post by Alan W. Irwin
Build Warnings (1)
C:\cmake-3.13.1-win64-x64\bin\cmake.exe

That's in the "Build" section and indicates that the build command
exited with non-zero status. If you want to try to reproduce that
by hand, run the `cmake --build . --config Debug` command rather
than `ctest`.
`cmake --build`'s exit code just forwards from the native build tool.
It may be that MSBuild has chosen to exist with non-zero for some reason.

Did you mean "ctest --build-and-test ..." rather than "cmake --build
..."? I assume if the build part of "ctest --build-and-test ..."
returned with some non-zero return code, then you would get a message
like the above that mentions ctest in a way that implies it is running
the show. But I cannot see any way you could get such a WARNING
message from "cmake --build ...". I hasten to add that I have been
completely content over the years to use cmake, make, and ctest
separately, and I have no experience using them in an
integrated way (e.g., "cmake --build ..." or "ctest --build-and-test
..."). So my comment is based on my quick reading the appropriate
sections of the man pages for cmake and ctest this morning, and I may
have missed some way that a cmake command could generate what appears
to be a message from ctest.

Never mind, I got it wrong. I was too focussed on the "ctest" in the
above message, and it pretty clearly was issued "from:
C:\cmake-3.13.1-win64-x64\bin\cmake.exe"

But I still don't understand exactly how ctest is run from cmake. Assuming the
"test" target (which I have never tried) runs the "all" target and then ctest,
I guess

cmake --build . --config Debug --target test

would run ctest indirectly after the "all" target was built, although
I haven't yet figured out how in this case that user specified the -D option to
submit a dashboard for that indirect ctest command. But even if that
puzzle can be solved, the above message does mention ctest so I am pretty sure the above
message is not coming from that "all" target. Which means
ctest itself is sending a non-zero return
code. And for the "--target test" case I don't see how that indirect
ctest command could actually build anything.

Anyhow, if you still feel the above rather opaque message was really
from a PLplot build error on this guy's platform, then it appears to
me (since both cmake and ctest are mentioned in the message) that it
must be from cmake calling ctest which calls the build tool, but I
don't see how that is possible.

Alan
__________________________
Alan W. Irwin

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers