Bug 116131

Summary:	"Directory does not exist" when doing tools perf_install out-of-tree build
Product:	Tools	Reporter:	TJ (linux)
Component:	Other	Assignee:	Tools.Other (tools_other)
Status:	NEW ---
Severity:	normal	CC:	eadams
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	v4.5+	Subsystem:
Regression:	No	Bisected commit-id:

Description TJ 2016-04-10 20:15:55 UTC

I have been attempting to build/install the perf tools in my usual out-of-tree build configuration but am getting:

output directory "/home/all/SourceCode/linux/builds/amd64/perf/perf/" does not exist

This looks to be a bug which triggers the error report in tools/scripts/Makefile.include on line 16:

$(if $(OUTDIR),, $(error output directory "$(OUTPUT)" does not exist))

The value of OUTPUT is set a few lines before by:

  OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)


My directory organisation:

source: /home/all/SourceCode/linux/linux
build:  /home/all/SourceCode/linux/builds/amd64/


To get to this stage I also had to fix another bug where the Documentation/ directory is expected to be found in the out-of-tree build tree.

$ ln -s ../../linux/Documentation ../builds/amd64/Documentation


tj@hephaestion:/home/all/SourceCode/linux/linux$

$ make -j 1 -C tools/ O=/home/all/SourceCode/linux/builds/amd64/ perf_install                                  

make: Entering directory '/home/all/SourceCode/linux/linux/tools'
  DESCEND  perf
make[1]: Entering directory '/home/all/SourceCode/linux/linux/tools/perf'
  BUILD:   Doing 'make -j2' parallel build

Auto-detecting system features:
...                         dwarf: [ on  ]
...                         glibc: [ on  ]
...                          gtk2: [ on  ]
...                      libaudit: [ on  ]
...                        libbfd: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ OFF ]
...        numa_num_possible_cpus: [ OFF ]
...                       libperl: [ OFF ]
...                     libpython: [ on  ]
...                      libslang: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]

config/Makefile:462: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev
config/Makefile:608: No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev
  GEN      perf-archive
  GEN      perf-with-kcore
  SUBDIR   Documentation
  ASCIIDOC /home/all/SourceCode/linux/builds/amd64/Documentation/perf-diff.xml
/bin/sh: 1: cd: can't cd to /home/all/SourceCode/linux/builds/amd64/perf/perf/
../../scripts/Makefile.include:16: *** output directory "/home/all/SourceCode/linux/builds/amd64/perf/perf/" does not exist.  Stop.
Makefile.perf:444: recipe for target '/home/all/SourceCode/linux/builds/amd64/perf//../lib/api/libapi.a' failed
make[2]: *** [/home/all/SourceCode/linux/builds/amd64/perf//../lib/api/libapi.a] Error 2
make[2]: *** Waiting for unfinished jobs....

Comment 1 TJ 2016-04-10 20:35:10 UTC

remake (the make debugger) shows:

$ remake -x -j 1 -C tools/ O=/home/all/SourceCode/linux/builds/amd64/ perf_install
...
  GEN      perf-archive
  GEN      perf-with-kcore
  SUBDIR   Documentation
  INSTALL  Documentation-man
/bin/sh: 1: cd: can't cd to /home/all/SourceCode/linux/builds/amd64/perf/perf/
../../scripts/Makefile.include:16: *** output directory "/home/all/SourceCode/linux/builds/amd64/perf/perf/" does not exist.  Stop.
Command-line invocation:
        "remake -C /home/all/SourceCode/linux/linux/tools/lib/api/ O=/home/all/SourceCode/linux/builds/amd64/perf/ /home/all/SourceCode/linux/builds/amd64/perf/libapi.a"
Makefile.perf:427: *** [/home/all/SourceCode/linux/builds/amd64/perf//../lib/api/libapi.a] Error 2

#0  /home/all/SourceCode/linux/builds/amd64/perf/libtraceevent.a at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:427
#1  /home/all/SourceCode/linux/builds/amd64/perf/perf at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:332
#2  all at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:4
#3  install-tools at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:539
#4  install-bin at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:593
#5  install at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:595
remake[2]: *** Waiting for unfinished jobs....
Command-line invocation:
        "remake -f Makefile.perf --no-print-directory -j2 O=/home/all/SourceCode/linux/builds/amd64 install"
Makefile:86: *** [install] Error 2

#0  install at /home/all/SourceCode/linux/linux/tools/perf/Makefile:86
Command-line invocation:
        "remake O=/home/all/SourceCode/linux/builds/amd64 subdir=perf --no-print-directory -C perf install"
Makefile:96: *** [perf_install] Error 2

#0  perf_install at /home/all/SourceCode/linux/linux/tools/Makefile:96
remake: Leaving directory `/home/all/SourceCode/linux/linux/tools'
Command-line invocation:
        "remake -x -j 1 -C tools/ O=/home/all/SourceCode/linux/builds/amd64/ perf_install"

Comment 2 TJ 2016-04-10 21:21:36 UTC

I tried to go back to calling the make target from the base source directory but still have a similar issue, albeit a slightly different path failure, although it may be easier to figure out now.

I also added $(if 1,$(info VARIABLE_NAME=$(VARIABLE_NAME))) into tools/scripts/Makefile.include to display the calculated values of the variables, which seems revealing.

It seems at some point the value of O is changed from the base out-of-tree dir to

O=/home/all/SourceCode/linux/builds/amd64/tools/perf/

which when the sub-path tools/perf/ is tacked on becomes:

OUTPUT=/home/all/SourceCode/linux/builds/amd64/tools/perf/tools/perf/


$ remake -x -j 1  O=/home/all/SourceCode/linux/builds/amd64/ tools/perf_install |& tee /tmp/make.log
...
O=/home/all/SourceCode/linux/builds/amd64
dummy=
ABSOLUTE_O=/home/all/SourceCode/linux/builds/amd64
OUTPUT=/home/all/SourceCode/linux/builds/amd64/Documentation/
COMMAND_O=O=/home/all/SourceCode/linux/builds/amd64
OUTDIR=/home/all/SourceCode/linux/linux/Documentation
  INSTALL  Documentation-man
O=/home/all/SourceCode/linux/builds/amd64/tools/perf/
dummy=
ABSOLUTE_O=/home/all/SourceCode/linux/builds/amd64/tools/perf
OUTPUT=/home/all/SourceCode/linux/builds/amd64/tools/perf/tools/perf/
COMMAND_O=O=/home/all/SourceCode/linux/builds/amd64/tools/perf
/bin/sh: 1: cd: can't cd to /home/all/SourceCode/linux/builds/amd64/tools/perf/tools/perf/
OUTDIR=
../../scripts/Makefile.include:22: *** output directory "/home/all/SourceCode/linux/builds/amd64/tools/perf/tools/perf/" does not exist.  Stop.
Command-line invocation:
        "remake -C /home/all/SourceCode/linux/linux/tools/lib/api/ O=/home/all/SourceCode/linux/builds/amd64/tools/perf/ /home/all/SourceCode/linux/builds/amd64/tools/perf/libapi.a"
Makefile.perf:427: *** [/home/all/SourceCode/linux/builds/amd64/tools/perf//../lib/api/libapi.a] Error 2

#0  /home/all/SourceCode/linux/builds/amd64/tools/perf/libtraceevent.a at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:427
#1  /home/all/SourceCode/linux/builds/amd64/tools/perf/perf at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:332
#2  all at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:4
#3  install-tools at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:539
#4  install-bin at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:593
#5  install at /home/all/SourceCode/linux/linux/tools/perf/Makefile.perf:595

Comment 3 TJ 2016-04-10 21:36:44 UTC

git blame tools/scripts/Makefile.include:

9e4a6648 (Borislav Petkov   2013-02-20 16:32:29 +0100  1) ifneq ($(O),)
bf35182f (David Howells     2012-11-05 21:02:08 +0000  2) ifeq ($(origin O), command line)
c883122a (Steven Rostedt    2012-08-13 10:23:02 -0400  3)       dummy := $(if $(shell test -d $(O) || echo $(O)),$(error O=$(O) does not exist),)
00000000 (Not Committed Yet 2016-04-10 22:35:18 +0100  4)       __T := $(if 1,$(info O=$(O)),)
00000000 (Not Committed Yet 2016-04-10 22:35:18 +0100  5)       __T := $(if 1,$(info dummy=$(dummy)),)
c883122a (Steven Rostedt    2012-08-13 10:23:02 -0400  6)       ABSOLUTE_O := $(shell cd $(O) ; pwd)
00000000 (Not Committed Yet 2016-04-10 22:35:18 +0100  7)       __T := $(if 1,$(info ABSOLUTE_O=$(ABSOLUTE_O)),)
bf35182f (David Howells     2012-11-05 21:02:08 +0000  8)       OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
00000000 (Not Committed Yet 2016-04-10 22:35:18 +0100  9)       __T := $(if 1,$(info OUTPUT=$(OUTPUT)),)
c883122a (Steven Rostedt    2012-08-13 10:23:02 -0400 10)       COMMAND_O := O=$(ABSOLUTE_O)
00000000 (Not Committed Yet 2016-04-10 22:35:18 +0100 11)       --T := $(if 1,$(info COMMAND_O=$(COMMAND_O)),)

Comment 4 TJ 2016-04-11 19:34:31 UTC

The broken tools/perf seems to be a result of the more recent changes that alter O before the tools/perf/Makefile.perf and explains why the O= path being passed already had tools/perf/ suffix:

commit 16671c1e1
Author: Jiri Olsa <jolsa@kernel.org>
Date:   Sat Apr 18 22:34:39 2015 +0200

    tools build: Fix Makefile(s) to properly invoke tools build

...
--- a/tools/Makefile
+++ b/tools/Makefile
...
@@ -50,8 +55,13 @@ liblockdep: FORCE
 libapi: FORCE
        $(call descend,lib/api)
 
+# The perf build does not follow the descend function setup,
+# invoking it via it's own make rule.
+PERF_O   = $(if $(O),$(O)/tools/perf,)
+
 perf: FORCE
-       $(call descend,$@)
+       $(Q)mkdir -p $(PERF_O) .
+       $(Q)$(MAKE) --no-print-directory -C perf O=$(PERF_O) subdir=
 
 selftests: FORCE
        $(call descend,testing/$@)