[cmake-developers] CMake strips MAKEFLAGS from environment

[Taylor Braun-Jones]

This is what I'm seeing:

$ echo 'message("MAKEFLAGS: $ENV{MAKEFLAGS}\nFOO: $ENV{FOO})"' >
CMakeLists.txt
$ MAKEFLAGS=test FOO=bar cmake .
-- The C compiler identification is GNU 4.8.3
-- The CXX compiler identification is GNU 4.8.3
-- Check for working C compiler: /usr/lib64/ccache/cc
-- Check for working C compiler: /usr/lib64/ccache/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/lib64/ccache/c++
-- Check for working CXX compiler: /usr/lib64/ccache/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
MAKEFLAGS:
FOO: bar
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/test_env_makeflags

Why is CMake stripping MAKEFLAGS from its environment? I couldn't find any
documentation describing this as normal/expected behavior.

Thanks,
Taylor

[Brad King]

Post by Taylor Braun-Jones
Why is CMake stripping MAKEFLAGS from its environment?

This has been done since our very early days in this commit:

BUG: Need to remove the MAKEFLAGS when cmake starts. If cmake is run from
inside make, we don't want the try-compiles to inherit the makeflags.
http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=02f7cfbc

What are you trying to do?

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[Taylor Braun-Jones]

Post by Brad King
What are you trying to do?

I'm trying to implement a workaround solution for setting environment
variables in my Makefiles at build time. So what I started with was this:

set(bullseye_environment COVFILE=${PROJECT_BINARY_DIR}/bullseye/test.cov)
add_custom_target(coverage_data
COMMAND env ${bullseye_environment}
${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR} --clean-first
COMMAND env ${bullseye_environment} ${CMAKE_CTEST_COMMAND}
--output-on-failure
COMMENT "Generating test coverage data"
VERBATIM
)

Which works fine for Ninja generator. It works for Unix Makefiles too,
except that the parallelism is lost. I get errors like:

gmake[1]: warning: jobserver unavailable: using -j1. Add `+' to parent
make rule.

So I added another workound for that:

if(${CMAKE_GENERATOR} MATCHES ".* Makefiles$")
set(environment_build_flags "$ENV{MAKEFLAGS}")
endif()

add_custom_target(coverage_data
COMMAND env --unset=MAKEFLAGS ${bullseye_environment}
${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR} --clean-first --
${environment_build_flags}
COMMAND env ${bullseye_environment} ${CMAKE_CTEST_COMMAND}
--output-on-failure
COMMENT "Generating test coverage data"
VERBATIM
)

Which allows the parallelism to work properly in an environment like:

export MAKEFLAGS=$(grep -i -c ^processor /proc/cpuinfo)
make coverage_data

Taylor

[Ben Boeckel]

Post by Taylor Braun-Jones
if(${CMAKE_GENERATOR} MATCHES ".* Makefiles$")
set(environment_build_flags "$ENV{MAKEFLAGS}")
endif()
add_custom_target(coverage_data
COMMAND env --unset=MAKEFLAGS ${bullseye_environment}
${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR} --clean-first --
${environment_build_flags}
COMMAND env ${bullseye_environment} ${CMAKE_CTEST_COMMAND}
--output-on-failure
COMMENT "Generating test coverage data"
VERBATIM
)
export MAKEFLAGS=$(grep -i -c ^processor /proc/cpuinfo)
make coverage_data

The problem with MAKEFLAGS is that `-i` also gets inherited and that is
not acceptable to use when running things like try_compile since it
causes false positives (e.g., detecting NEON extensions on x86). CMake
clears it so that it has a better expectation of how things will run. Is
there a reason you can't use a CMake cache variable to set
`environment_build_flags` rather than the environment (since it would
get remembered between cmake runs while the environment needs to be
specified every time)?

--Ben
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[Taylor Braun-Jones]

Post by Ben Boeckel
The problem with MAKEFLAGS is that `-i` also gets inherited and that is
not acceptable to use when running things like try_compile since it
causes false positives (e.g., detecting NEON extensions on x86). CMake
clears it so that it has a better expectation of how things will run. Is
there a reason you can't use a CMake cache variable to set
`environment_build_flags` rather than the environment (since it would
get remembered between cmake runs while the environment needs to be
specified every time)?

Ya, I could definitely use a cache variable and that probably makes sense
to do here. But I'm not sure how it solves my problem. For example, with
this CMakeLists.txt:

cmake_minimum_required(VERSION 3.0)
set(environment_build_flags $ENV{MAKEFLAGS} CACHE STRING "MAKEFLAGS")
add_custom_target(echo_makeflags COMMAND echo "environment_build_flags:
${environment_build_flags}" VERBATIM)

I still just get:

$ make echo_makeflags
environment_build_flags:
Built target echo_makeflags

Taylor

[Brad King]

gmake[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.

I don't think that is caused by the MAKEFLAGS removal I linked.
The code I linked does not run during 'cmake --build'. It only
runs while configuring a project to protect try_compile.

The problem is that make does not pass MAKEFLAGS when invoking a tool
it does not think is 'make'. For example:

-----------------------------------------------------------------------------
$ cat env.make
all:
@env make -f env.make other

other:
@echo MAKEFLAGS='$(MAKEFLAGS)'

$ make -j2 -f env.make other
MAKEFLAGS= -j --jobserver-fds=3,4

$ make -j2 -f env.make all
make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
MAKEFLAGS=w
-----------------------------------------------------------------------------

You could try using "+env" as your COMMAND instead, though I wouldn't
consider this an officially supported approach.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[Taylor Braun-Jones]

Post by Brad King
You could try using "+env" as your COMMAND instead, though I wouldn't
consider this an officially supported approach.

Thanks - that works and is a much better workaround.

[Brad King]

I'm trying to implement a workaround solution for setting environment variables
set(bullseye_environment COVFILE=${PROJECT_BINARY_DIR}/bullseye/test.cov)
add_custom_target(coverage_data
COMMAND env ${bullseye_environment}
${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR} --clean-first
COMMAND env ${bullseye_environment} ${CMAKE_CTEST_COMMAND} --output-on-failure

We typically run coverage as part of nightly testing with a 'ctest -S'
dashboard client script. The script surrounds the actual build
invocation and can take responsibility for adding things like COVFILE
to the environment.

-Brad
--
Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[Taylor Braun-Jones]

Post by Brad King
We typically run coverage as part of nightly testing with a 'ctest -S'
dashboard client script. The script surrounds the actual build
invocation and can take responsibility for adding things like COVFILE
to the environment.

Ya, that was one of my first thoughts and it does make things simpler. But
I wanted developers to be able to get instant coverage feedback at their
desktop to shorten the write test/check coverage cycle. We have high MC/DC
coverage goals and as you start to approach 100% coverage it's not always
obvious what conditions/decisions are the missing coverage. I also want to
keep the usage simple and consistent meaning that code coverage and static
analysis reports should be available as a build target (e.g. `make
coverage_report coverity_report`)

Taylor