[cmake-developers] Python bindings for CTest

[Jonathan R. Madsen]

Hi,

I have created some Python bindings for CTest. The goal was to create an easy way for Python projects that I work with to be able to wrap their Python compilation log (if there was one), dynamically generate CTests and/or wrap their existing testing commands, do any additional testing analysis (in Python), etc. and then submit the logs, notes, plots, etc. to CDash without requiring them to overhaul any of their existing build or testing systems.

Is there any interest in this development?

Jonathan
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Brad King]

Post by Jonathan R. Madsen
I have created some Python bindings for CTest. The goal was to
create an easy way for Python projects that I work with to be able
to wrap their Python compilation log (if there was one), dynamically
generate CTests and/or wrap their existing testing commands, do any
additional testing analysis (in Python), etc. and then submit the
logs, notes, plots, etc. to CDash without requiring them to overhaul
any of their existing build or testing systems.
Is there any interest in this development?

Neat. Have you published these anywhere?

Does this make modifications to CTest itself, or is this all
going through CTestTestfile.cmake generation? I don't think we
can have "ctest" itself link to python libraries.

Thanks,
-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[Jonathan R. Madsen]

Inline below.

--
Jonathan

Post by Brad King

Post by Jonathan R. Madsen
I have created some Python bindings for CTest. The goal was to
create an easy way for Python projects that I work with to be able
to wrap their Python compilation log (if there was one), dynamically
generate CTests and/or wrap their existing testing commands, do any
additional testing analysis (in Python), etc. and then submit the
logs, notes, plots, etc. to CDash without requiring them to overhaul
any of their existing build or testing systems.
Is there any interest in this development?

Neat. Have you published these anywhere?

Yes it is currently available at https://github.com/jrmadsen/pyctest.
The 'conda-fixes' branch is what I've been working on most recently but I hasn't diverged much outside of the examples. This is a good example of how the driver works:

https://github.com/jrmadsen/pyctest/blob/conda-fixes/examples/TomoPy/pyctest_tomopy.py

Post by Brad King
Does this make modifications to CTest itself,

It does not modify CTestLib or CMakeLib in any substantial way. I had to reimplement some classes to bypass some expectations -- such as the expections for actual files -- but when that was required I prefixed the new/alternative implementation with "py" (e.g. cmConfigureFileCommand became pycmConfigureFileCommand).

Post by Brad King
or is this all
going through CTestTestfile.cmake generation?

I'm not 100% sure what you are asking here. For the most part, I let y'all's library do all the work. For example, the Python version of cmTest (pycmTest) uses cmPropertyMap and the Python version of cmTestGenerator (pycmTestGenerator) inherits from cmScriptGenerator. I believe this to be a far less error-prone method than just creating a Python package that "knows the syntax" and writes files for CMake/CTest.

Post by Brad King
I don't think we
can have "ctest" itself link to python libraries.

Nothing on the CMake side has any dependencies on python libraries as a result of this. I just include header files and link to y'all's library like one does with any other API.

Post by Brad King
Thanks,
-Brad

[Brad King]

Post by Jonathan R. Madsen
I just include header files and link to y'all's library like one
does with any other API. 

As a matter of policy we do not offer any stable API, SDK, headers,
or libraries to be linked for exposing CMake internals. Any external
package that tries to do this must take responsibility for updating
as we make internal changes.

The syntax of CTestTestfile.cmake files is much more stable.

If your goal is to submit to CDash, one could generate the .xml
files directly from python and not need CTest at all.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
h

[Ben Boeckel]

Post by Brad King
If your goal is to submit to CDash, one could generate the .xml
files directly from python and not need CTest at all.

There is some code in Spack which does this already (though it is very
Spack-specific, it might provide some help):

https://github.com/spack/spack/blob/1e1ea54bada54ae783198823c7f8fe0430ac41cb/lib/spack/spack/reporters/cdash.py
https://github.com/spack/spack/tree/e5a68327608d6a5e6c0709237517238e0de96c27/share/spack/templates/reports/cdash

--Ben
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers