Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPP gdb lldb no debugging symbols found #2537

Open
unship opened this issue Feb 16, 2017 · 44 comments
Open

CPP gdb lldb no debugging symbols found #2537

unship opened this issue Feb 16, 2017 · 44 comments
Labels
not stale Issues or PRs that are inactive but not considered stale P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug

Comments

@unship
Copy link

unship commented Feb 16, 2017

Please provide the following information. The more we know about your system and use case, the more easily and likely we can help.

Description of the problem / feature request / question:

i build CPP with bazel on my mac, when i debug it

bazel build -c dbg :main
gdb ./bazel-bin/main

it came out:

/main.pic.o': can't open to read symbols: No such file or directory.
(no debugging symbols found)...done.

If possible, provide a minimal example to reproduce the problem:

#WORKSPACE empty
#BUILD 
cc_binary(
 name='main',
srcs=['src/main.cc']
)
#src/main.cc
#include<iostream>
int main(){
  std::cout<<"test";
}
bazel build -c dbg :main
gdb ./bazel-bin/main

Environment info

  • Operating System:
    mac 10.12.3

  • Bazel version (output of bazel info release):
    release 0.4.0-homebrew

  • If bazel info release returns "development version" or "(@non-git)", please tell us what source tree you compiled Bazel from; git commit hash is appreciated (git rev-parse HEAD):

Have you found anything relevant by searching the web? (e.g. GitHub issues, email threads in the [email protected] archive)

no

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

FYI:
on ubuntu, gdb work is fine, but lldb still can't find debugging symbols

@dslomov dslomov added category: rules > C++ type: bug P2 We'll consider working on this in future. (Assignee optional) labels Feb 16, 2017
@unship
Copy link
Author

unship commented Feb 17, 2017

dSYM is generated,but is is empty, even when add --strip never

/Users/liyanan/dev/clion/bazel-bin/main empty dSYM file detected, dSYM was created with an executable with no debug info

@david-german-tri
Copy link

david-german-tri commented Feb 18, 2017

RobotLocomotion/drake developers who use OS X are experiencing this also. We are actively investigating workarounds. If we find something, I'll update this thread, but any guidance from the Bazel team would be hugely appreciated!

@david-german-tri
Copy link

Related: #327.

I'm having some preliminary success running dsymutil from osx_cc_wrapper.sh. With that technique, I can at least produce a usable .dSYM somewhere deep in the sandbox, then manually symbolicate.

@david-german-tri
Copy link

david-german-tri commented Feb 21, 2017

Here's an outline of a workaround we've prototyped locally:

  1. For every cc_{library,binary,test}, add a genrule like the following:
genrule(
    name = "foo_dsym",
    srcs = [":foo"],
    outs = ["foo.dSYM"],
    output_to_bindir = True,
    cmd = "dsymutil $(location :foo) -o $@"
)

We have binary/library/test wrapper macros that simplify insertion of this genrule across the board.

  1. Use --spawn_strategy=standalone to disable sandboxing, so that the DWARF-laden .o files will be visible to dsymutil. (Also, of course, use -c dbg, which in turn sets the compiler flag -g.)

  2. If you need to support platforms other than OS X, use a select statement on the cmd, and simply touch $@ on non-Apple platforms.

This has a bunch of deficiencies. It disables sandboxing, it declares a directory output from a genrule (which is unsound), it clutters the build with extra targets, and it doesn't actually build the debug symbols on bazel build :foo. But it at least provides a way to get debug symbols on OS X. If someone else has a better way, I'd be very happy!

@hlopko hlopko self-assigned this Feb 24, 2017
david-german-tri pushed a commit to david-german-tri/drake that referenced this issue Feb 27, 2017
david-german-tri pushed a commit to david-german-tri/drake that referenced this issue Feb 27, 2017
This generates .dSYM files for cc_binary and cc_test rules.

See bazelbuild/bazel#2537
kunimatsu-tri pushed a commit to kunimatsu-tri/drake that referenced this issue Mar 1, 2017
This generates .dSYM files for cc_binary and cc_test rules.

See bazelbuild/bazel#2537
jadecastro pushed a commit to jadecastro/drake that referenced this issue Mar 1, 2017
This generates .dSYM files for cc_binary and cc_test rules.

See bazelbuild/bazel#2537
@sistr22
Copy link

sistr22 commented Jul 7, 2017

Is the workaround above still the one we should use ?

@hlopko
Copy link
Member

hlopko commented Jul 7, 2017

CC @c-parsons @calpeyser

@yageek
Copy link

yageek commented Aug 10, 2017

@david-german-tri Would it be possible to give some more explanation on how to use the workaround ? I added a genrule after a cc_library library statement but I still get an empty .dSYM error. Should we also add --strip=never to the build/test command ?

@dattanchu
Copy link

I use the workaround and can generate the dSYM required to step through the executable with lldb. When we run in sandbox mode, is it possible to access the object files required by dsymutil?

@baskus
Copy link

baskus commented Jan 21, 2018

What is the status of this? When will we get debug symbols without hacks?

Thanks! :)

@jmarantz
Copy link

jmarantz commented Feb 8, 2018

+1 -- I had the misfortune of introducing a problem that could be repro'd only on Mac and the question was whether to deal with this hack, or just use printfs. I used printfs, but I next time I can just debug on mac.

dnoe pushed a commit to envoyproxy/envoy that referenced this issue Feb 8, 2018
Description:
bombela/backward-cpp now has some support for backtraces on OS X. Bump backward-cpp to a revision that contains the change. Modify stack trace output on OS X since addr2line (via homebrew's binutils package) doesn't seem to be able to resolve addresses from the stack trace (or lldb for that matter). Instead, print as much information as possible directly in the stack trace (which is equivalent to lldb's output given bazelbuild/bazel#2537).

Risk Level: Low

Testing:
Manually tested stack traces by forcing division by zero.

Signed-off-by: Stephan Zuercher [email protected]
@steve-the-bayesian
Copy link
Contributor

+1. This is a major pain point for me. Thanks to everyone who's working to fix it.

@tony-shi
Copy link

+1. BTW, will this issue be fixed/upgrade officially or we have to take a hack described above to enable generate debugging symbols?
Thanks!

@DavidGoldman
Copy link
Contributor

rules_apple supports building dSYMs inside the linking action itself (like @haberman mentioned), although this means slowing down your critical path. If you ever build remotely you'll need dSYMs in order to debug unless your machine's paths matches the builders' paths exactly.

If you don't use dSYMs you'll need to disable sandboxing for the linking step, e.g. setting --strategy=ObjcLink=standalone and --strategy=CppLink=standalone. Like @haberman this wouldn't be a problem if the absolute paths to the .o files were still valid after the linking action finishes.

@wesleyw72
Copy link

Hi @hlopko , even with setting --spawn_strategy=local and -c dbg I still can’t get source level debugging. For example typing l <source file path> doesn’t work.

@hlopko
Copy link
Member

hlopko commented Nov 25, 2019

@wesleyw72 I'm afraid 'll need a repro case for that, it should work AFAIK. Pls include Bazel version and all Bazel options you pass (don't forget the .bazelrc).

@wesleyw72
Copy link

@hlopko I think the issue I was hitting is that the cache was being hit, so changing the spawn_strategy wasn't making a difference as no actions were being spawned and thus the object files with the debugging symbols were missing (in a previously cleaned up sandbox). If I do a clean and then build with spawn_strategy=local then it does seem to be working. Thanks!

@nikhilm
Copy link
Contributor

nikhilm commented Sep 30, 2020

Trying to understand exactly what the deficiency here is, because it seems the macos_* rules from rules_apple can build dsyms.
Specifically, if one wants a dylib on macOS

cc_binary(
  name = "libfoo.dylib",
)

will not produce a dsym (nor will dwarf debug info be preserved in the dylib itself)

but

cc_library(
  name = "foo",
)

macos_dylib(
  name = "libfoo",
  deps = [":foo"],
  ...
)

when built with --apple_generate_dsym will generate valid dSYMs. I also see that wrapped_clang has a notion of DSYM_HINT_DSYM_PATH that is only set for objc targets in

"DSYM_HINT_DSYM_PATH=%{dsym_path}",
. Would extending that to also work for C/C++ executable actions be sufficient?

@haberman
Copy link
Contributor

haberman commented Oct 7, 2020

@nikhilm I am concern about the statically-linked binary case, not dylib.

In my view, this bug is fixed when I can:

  1. build a cc_binary() in Bazel using -c opt or --copt=-g
  2. run the binary in gdb/lldb/Instruments/etc
  3. have full debugging symbols available.

When I do this today, I get no debugging symbols available.

@keith
Copy link
Member

keith commented Oct 7, 2020

dSYMs are the solution for both of those cases. A workaround on macOS could actually even be to build a macos_command_line_application just for the dSYM support.

@c-mita c-mita added P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) and removed P2 We'll consider working on this in future. (Assignee optional) labels Nov 24, 2020
bazel-io pushed a commit that referenced this issue Oct 19, 2021
Expands the targeted actions to allow stripping the absolute build path for debug symbols from all libraries.  The support in wrapped_clang is indifferent to whether we are linking a true objc or regular cpp library.

When used in concert with `--features=oso_prefix_is_pwd --remote_download_outputs=all`, allows successful local debugging of executables/libraries built remotely.  Probably also helps with sandboxed actions.

Helps mitigate #6327 and kinda sorta #2537?  See also #11671

Closes #13311.

PiperOrigin-RevId: 404262861
@bedbad
Copy link

bedbad commented Feb 11, 2022

As of right now I can confirm that when you build using "bazel build -c dbg --spawn_strategy=local" on Apple M1 ("darwin"), after clean, you can not find debug symbols in the binary.

@keith
Copy link
Member

keith commented Feb 11, 2022

Since my previous comment the dSYM logic also applies to cc_binary targets now, can you pass --apple_generate_dsym and see if that fixes your issues?

@bedbad
Copy link

bedbad commented Feb 12, 2022

My bad. It does put the debug symbols in binary with the --spawn_strategy=local on Apple M1 Sillicon
[My eye glanced over the small "-c opt" flag further in the invocation].
Since this thread is still fairly active, I'd add the general method for debugging bazel flag issues like these which in my experience is most efficient (and get some bazel team feedback), since I had to resolve a good mass of bazel issues porting large build on Apple Sillicon M1 ("darwin" - arm64). Which are still newish and by far the fastest personal machine I built on out of all.

    • run bazelisk ... with "-s" option (print the full command invocations).
      Identify the action of interest and its bazel type, note the full command invocation. and concrete arguments of interest. In this example "-g0", "-O2", "-DNDEBUG" (precluding clang debug symbols) and/or the wrapper/tool path bazel uses in it.
      Even if you try hard to pass the correct flags to bazel they are still be copied later in invocation, which in compiler case (not in bazel which looks to be inverse case) means they are actually ignored. (which was the case above)
  1. Map these to the actual bazel configuration that runs in your case that you can find in ${output_base}/
    bazelisk info
...
output_base: /private/var/tmp/_bazel_im/b5fbef499f0aa54a50d4f7f63c872539
...

And search the actual code in places like ${output_base}/install/embedded_tools/tools/[ cpp| osx | ...]
Where your flags of interest originate from.
install/embedded_tools/tools/cpp/cc_toolchain_config.bzl:

   526	    if (ctx.attr.cpu == "darwin"):

   527	        default_compile_flags_feature = feature(
   528	            name = "default_compile_flags",
   529	            enabled = True,
   530	            flag_sets = [
   531	                flag_set(
   532	                    actions = [
   533	                        _ASSEMBLE_ACTION_NAME,
   534	                        _PREPROCESS_ASSEMBLE_ACTION_NAME,
   535	                        _LINKSTAMP_COMPILE_ACTION_NAME,
   536	                        _C_COMPILE_ACTION_NAME,
   537	                        _CPP_COMPILE_ACTION_NAME,
   538	                        _CPP_HEADER_PARSING_ACTION_NAME,
   539	                        _CPP_MODULE_COMPILE_ACTION_NAME,
   540	                        _CPP_MODULE_CODEGEN_ACTION_NAME,
   541	                        _LTO_BACKEND_ACTION_NAME,
   542	                        _CLIF_MATCH_ACTION_NAME,
   543	                    ],
   544	                    flag_groups = [
   545	                        flag_group(
   546	                            flags = [
   547	                                "-D_FORTIFY_SOURCE=1",
   548	                                "-fstack-protector",
   549	                                "-fcolor-diagnostics",
   550	                                "-Wall",
   551	                                "-Wthread-safety",
   552	                                "-Wself-assign",
   553	                                "-fno-omit-frame-pointer",
   554	                            ],
   555	                        ),
   556	                    ],
   557	                ),
   558	                flag_set(
   559	                    actions = [
   560	                        _ASSEMBLE_ACTION_NAME,
   561	                        _PREPROCESS_ASSEMBLE_ACTION_NAME,
   562	                        _LINKSTAMP_COMPILE_ACTION_NAME,
   563	                        _C_COMPILE_ACTION_NAME,
   564	                        _CPP_COMPILE_ACTION_NAME,
   565	                        _CPP_HEADER_PARSING_ACTION_NAME,
   566	                        _CPP_MODULE_COMPILE_ACTION_NAME,
   567	                        _CPP_MODULE_CODEGEN_ACTION_NAME,
   568	                        _LTO_BACKEND_ACTION_NAME,
   569	                        _CLIF_MATCH_ACTION_NAME,
   570	                    ],
   571	                    flag_groups = [flag_group(flags = ["-g"])],
   572	                    with_features = [with_feature_set(features = ["dbg"])],
   573	                ),
   574	                flag_set(
   575	                    actions = [
   576	                        _ASSEMBLE_ACTION_NAME,
   577	                        _PREPROCESS_ASSEMBLE_ACTION_NAME,
   578	                        _LINKSTAMP_COMPILE_ACTION_NAME,
   579	                        _C_COMPILE_ACTION_NAME,
   580	                        _CPP_COMPILE_ACTION_NAME,
   581	                        _CPP_HEADER_PARSING_ACTION_NAME,
   582	                        _CPP_MODULE_COMPILE_ACTION_NAME,
   583	                        _CPP_MODULE_CODEGEN_ACTION_NAME,
   584	                        _LTO_BACKEND_ACTION_NAME,
   585	                        _CLIF_MATCH_ACTION_NAME,
   586	                    ],
   587	                    flag_groups = [
   588	                        flag_group(
   589	                            flags = [
   
   590	                                "-g0",
   591	                                "-O2",
   592	                                "-DNDEBUG",

   593	                                "-ffunction-sections",
   594	                                "-fdata-sections",
   595	                            ],
   596	                        ),
   597	                    ],
   598	                    with_features = [with_feature_set(features = ["opt"])],
   599	                ),
   600	                flag_set(
   601	                    actions = [
   602	                        _LINKSTAMP_COMPILE_ACTION_NAME,
   603	                        _CPP_COMPILE_ACTION_NAME,
   604	                        _CPP_HEADER_PARSING_ACTION_NAME,
   605	                        _CPP_MODULE_COMPILE_ACTION_NAME,
   606	                        _CPP_MODULE_CODEGEN_ACTION_NAME,
   607	                        _LTO_BACKEND_ACTION_NAME,
   608	                        _CLIF_MATCH_ACTION_NAME,
   609	                    ],
   610	                    flag_groups = [flag_group(flags = ["-std=c++0x"])],
   611	                ),
   612	            ],
   613	        )

  1. Understand the relevant code. Trace it upwards and so you can be sure what exactly bazel does in your case.
    For example, in Apple M1 debug case those arguments happen in darwin exclusively with_feature_set(features = ["opt"]) which categorically told me that "-c opt" is still somehow in my invocation(which was confusing because it looks like --copt)

This at least has the lowest guarantee on completion time which can be huge and has always improving learning curve. Although it can be much faster then try-balling from issues/SO for the particular complicated build which production bazel builds normally are. I don't really find that available documentation currently helps such issues much.

@constinit
Copy link

constinit commented Apr 1, 2022

I can see debug symbols (including line numbers) when building with -c dbg --spawn_strategy=local on an M1 Mac with homebrew llvm.

Full command:

bazel  test --config=asan -c dbg --spawn_strategy=local //...

@fansehep
Copy link

I meet thie problem in my archlinux too.
bazel version: 5.3.1
archlinux version: 5.19.3
gcc version: 12.2.0
gdb version: 12.1
Even if i use -dbg or debug mode useless.
but seems the gdb find the symbols. but don't display.
The gdb tui display Reading symbols from bazel-bin/src/base/test/..., but don;t display.

@umap341
Copy link

umap341 commented Oct 20, 2022

On windows also facing same issue: (No debugging symbols found in bazel-bin/examples/application/application). Any solution to this?

Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label Dec 25, 2023
@chandlerc
Copy link

Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

Trying @bazelbuild/triage -- this is definitely still relevant and should be resolved. It is quite easy to reproduce.

@chandlerc
Copy link

Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

Trying @bazelbuild/triage -- this is definitely still relevant and should be resolved. It is quite easy to reproduce.

And as I suspected, I can't actually follow the steps described here: typing @bazelbuild/triage does nothing for me. It is incredibly infuriating to have a bot auto-close issues in an open source repository with instructions that no reporter can actually follow for keeping issues open. =[

@fmeum
Copy link
Collaborator

fmeum commented Dec 25, 2023

@bazelbuild/triage

I remember someone from the team saying that secret teams can still be mentioned even if autocomplete doesn't work for them.

@github-actions github-actions bot removed the stale Issues or PRs that are stale (no activity for 30 days) label Dec 26, 2023
@sgowroji sgowroji added the not stale Issues or PRs that are inactive but not considered stale label Dec 26, 2023
@comius comius added P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) labels Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not stale Issues or PRs that are inactive but not considered stale P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-CPP Issues for C++ rules type: bug
Projects
None yet
Development

No branches or pull requests