[cmake-developers] Error handling in dashboard scripts

[Kyle Edwards]

Hi all,

I'm working on a set of build scripts that use CMake and CTest, and I'm
trying to figure out the best way to handle failures in CTest (I'm using
a dashboard script internally.) If the configure or build step fails, I
want the failure to be reported to CDash with ctest_submit(), but I also
want CTest to exit with an error code so that the calling process can
detect a failure.

The documentation for ctest_configure(), ctest_build(), and
ctest_submit() isn't completely clear on what happens if one of these
steps fails. Let's say I have the following dashboard script (this is
pseudocode, arguments have been deliberately omitted for brevity, this
example won't work):

ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()

What happens if ctest_configure() fails (for example, because CMake
failed to find a needed library)? Does the entire script stop right
there? Or does it continue and try to execute ctest_build() anyway? Does
ctest_build() silently do nothing because the configure step failed?
Looking through the documentation for ctest_configure() and
ctest_build(), I see some information on the RETURN_VALUE and
CAPTURE_CMAKE_ERROR arguments, but it's not clear what happens if these
aren't used.

If someone could clarify for me what's supposed to happen here, and what
the recommended best practices are for making sure that ctest_submit()
still gets called in the event of a failure, I will gladly submit a
documentation patch with this information. I'm guessing it involves
using the RETURN_VALUE and CAPTURE_CMAKE_ERROR arguments, but the
documentation doesn't state this.

Kyle
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Brad King]

Post by Kyle Edwards
ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()
What happens if ctest_configure() fails (for example, because CMake
failed to find a needed library)? Does the entire script stop right
there? Or does it continue and try to execute ctest_build() anyway?

IIRC it just moves on to each command in sequence and does what that
command would normally do, letting it fail as it may given whatever
state in the build tree is left behind by earlier failures, if any.

Unless the failure is so bad that ctest_submit hasn't been told
where to submit the results it should always be reached.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Shawn Waldon]

The documentation for ctest_configure(), ctest_build(), and ctest_submit()
isn't completely clear on what happens if one of these steps fails. Let's
say I have the following dashboard script (this is pseudocode, arguments
ctest_start()
ctest_configure()
ctest_build()
ctest_test()
ctest_submit()
What happens if ctest_configure() fails (for example, because CMake failed
to find a needed library)? Does the entire script stop right there? Or does
it continue and try to execute ctest_build() anyway? Does ctest_build()
silently do nothing because the configure step failed? Looking through the
documentation for ctest_configure() and ctest_build(), I see some
information on the RETURN_VALUE and CAPTURE_CMAKE_ERROR arguments, but it's
not clear what happens if these aren't used.
If someone could clarify for me what's supposed to happen here, and what
the recommended best practices are for making sure that ctest_submit()
still gets called in the event of a failure, I will gladly submit a
documentation patch with this information. I'm guessing it involves using
the RETURN_VALUE and CAPTURE_CMAKE_ERROR arguments, but the documentation
doesn't state this.

In our VTK/ParaView dashboards we ensure the submit happens by doing
partial submissions after each stage (this also gives incremental results
on CDash):

set(success TRUE)

ctest_configure(RETURN_VALUE configure_result)
ctest_submit(PARTS Configure) # Tell submit to submit a partial result with
only the configure step
if (configure_result)
message("Configure failed")
set(success FALSE)
# you could exit here
endif()

if (success)
... # run next stage here

HTH,
Shawn