Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

file based filter #99

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
c13cf11
file based filter
xinzhengzhang Dec 23, 2022
0ea3fd6
Unify --file parsing logic
cpsauer Dec 24, 2022
4244747
file based filter: deal with header lib
xinzhengzhang Mar 18, 2023
0befc09
End loop if header is found & Add max attempts limit
xinzhengzhang Mar 3, 2023
92e83c1
Improve my language for --file docstring
cpsauer Mar 18, 2023
bf0405e
rm dupe itertools import
cpsauer Mar 18, 2023
9d7f9ab
use variable above
xinzhengzhang Mar 27, 2023
9903410
check file if resolved in a smaller scope
xinzhengzhang Mar 27, 2023
d7e1e31
Improve --file editor wrapper message
cpsauer Apr 24, 2023
d6f60cb
Fix no output error message and spacing.
cpsauer Apr 24, 2023
2b69f05
Fix small old typo
cpsauer May 2, 2023
242d601
Prevent template substitution in comment
cpsauer May 2, 2023
e17428f
ws
cpsauer May 2, 2023
5be98b1
Further improve --file editor wrapper message
cpsauer May 2, 2023
2e980ab
Alphabetize threading import
cpsauer May 2, 2023
8d5f2fe
Document interaction of --file and exclude_* template arguments
cpsauer May 2, 2023
e288d49
Entangled cleanups around thread structure
cpsauer May 2, 2023
4a251e5
Reduce unnecessary work with in header finding with --file flag
cpsauer May 2, 2023
0d86038
improve comments in _get_command
cpsauer May 2, 2023
da95ea6
Mark problems, fix typos, fix depleted iterator bug
cpsauer May 2, 2023
2a10207
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 2, 2023
f4fa941
Fix aquery inputs() statement not working for headers
cpsauer May 3, 2023
2e10a53
Add a few more WIP notes
cpsauer May 3, 2023
0851bca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 3, 2023
0f120b6
Fix typo
cpsauer May 3, 2023
9d23866
Add note about allpaths issue
cpsauer May 3, 2023
2d09c75
Refine and note discussion points in _filter_through_headers
cpsauer May 4, 2023
0722615
Improve TODOs
cpsauer May 4, 2023
14e707e
Enrich todos and notes after another read through of the bazel [ac]qu…
cpsauer May 4, 2023
bd7bde1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 4, 2023
89f583a
refresh.template.py: Estimate the number of actions that can be run b…
xinzhengzhang May 5, 2023
ef33509
refresh.template.py: quoting target_statement
xinzhengzhang May 5, 2023
f8b046e
refersh.template.py: running a preliminary query to get specified fi…
xinzhengzhang May 5, 2023
9ce1716
refresh.template.py: simplify/unify logic around target_statement_can…
xinzhengzhang May 5, 2023
40b9131
refresh.template.py: changed the _convert_compile_commands to return …
xinzhengzhang May 5, 2023
6a6777a
normalize file path if the --file is a subpath of the cwd
xinzhengzhang May 5, 2023
0453be4
refresh.template.py: for --file, don't continue traversing targets af…
xinzhengzhang May 5, 2023
97644fd
Revert "refresh.template.py: for --file, don't continue traversing ta…
xinzhengzhang May 6, 2023
e8a986b
Revert "normalize file path if the --file is a subpath of the cwd"
xinzhengzhang May 6, 2023
e67c0bd
Revert "refresh.template.py: changed the _convert_compile_commands to…
xinzhengzhang May 6, 2023
b785c3e
Revert "refresh.template.py: simplify/unify logic around target_state…
xinzhengzhang May 6, 2023
a152055
Revert "refersh.template.py: running a preliminary query to get spec…
xinzhengzhang May 6, 2023
1d8f62e
Revert "refresh.template.py: quoting target_statement"
xinzhengzhang May 6, 2023
33b2310
Revert "refresh.template.py: Estimate the number of actions that can …
xinzhengzhang May 6, 2023
84822ba
Add link to deduped aspects cquery/aquery aspects issue
cpsauer May 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refresh.template.py: simplify/unify logic around target_statement_can…
…didates and warning
  • Loading branch information
xinzhengzhang committed May 5, 2023
commit 9ce17168b54a2838705114b94c28bba3fb3efb84
49 changes: 25 additions & 24 deletions refresh.template.py
Original file line number Diff line number Diff line change
Expand Up @@ -939,14 +939,8 @@ def _get_compile_commands_for_aquery(aquery_target_statement: str, additional_aq
return []

if not getattr(parsed_aquery_output, 'actions', None): # Unifies cases: No actions (or actions list is empty)
# TODO the different flags warning in stderr is common enough that we might want to either filter it or adjust this messaging.
# TODO simplify/unify logic around warning about not being able to find commands.
if aquery_process.stderr:
log_warning(f""">>> Bazel lists no applicable compile commands for {target}, probably because of errors in your BUILD files, printed above.
Continuing gracefully...""")
elif not focused_on_file:
log_warning(f""">>> Bazel lists no applicable compile commands for {target}
If this is a header-only library, please instead specify a test or binary target that compiles it (search "header-only" in README.md).
Continuing gracefully...""")
return []

Expand Down Expand Up @@ -988,12 +982,13 @@ def _get_commands(target: str, flags: str):


# Then, actually query Bazel's compile actions for that configured target
target_statement = f"deps('{target}')"
compile_commands = [] # TODO simplify loop, especially if we can reduce it to one command per case (see below)? Move warning messages outside?
target_statement_candidates = []
file_path = None
compile_commands = []

if file_flags:
file_path = file_flags[0]
found = False
target_statement_candidates = []
target_statement = f"deps('{target}')"
if file_path.endswith(_get_files.source_extensions):
target_statement_candidates.append(f"inputs('{re.escape(file_path)}', {target_statement})")
else:
Expand All @@ -1009,24 +1004,30 @@ def _get_commands(target: str, flags: str):
f"allpaths({target}, {header_target_statement})", # Ordering is ideal, breadth-first from the deepest dependency, despite the docs. TODO (1) There's a bazel bug that produces extra actions, not on the path but downstream, so we probably want to pass --noinclude_aspects per https://github.com/bazelbuild/bazel/issues/18289 to eliminate them (at the cost of some valid aspects). (2) We might want to benchmark with --infer_universe_scope (if supported) and --universe-scope=target with query allrdeps({header_target_statement}, <maybe some limited depth>) or rdeps, checking speed but also ordering (the docs indicate it is likely to be lost, which is a problem) and for inclusion of the header target. We'd guess it'll have the same aspects bug as allpaths. (3) We probably also also want to *just* run this query, not the whole list, since it captures the former and is therefore unlikely to add much latency, since a given header is probabably either used internally to the target (find on first match) for header-only (must traverse all paths in all targets until you get a match) for all top-level targets, and since we can separate out the last, see below.
f'deps({target})', # TODO: Let's detect out-of-bazel, absolute paths and run this if and only if we're looking for a system header. We need to think about how we want to handle absolute paths more generally, perhaps normalizing them to relative if possible, like with the windows absolute path issue, above.
])
for target_statement in target_statement_candidates:
commands = list(_get_compile_commands_for_aquery(target_statement, additional_flags, file_path))
compile_commands.extend(commands) # If we did the work to generate a command, we'll update it, whether it's for the requested file or not.
if any(command['file'].endswith(file_path) for command in commands):
found = True
break
if not found:
log_warning(f""">>> Couldn't quickly find a compile command for {file_path} in {target}
Continuing gracefully...""") # TODO simplify/unify logic around warning about not being able to find commands.
else:
if {exclude_external_sources}:
# For efficiency, have bazel filter out external targets (and therefore actions) before they even get turned into actions or serialized and sent to us. Note: this is a different mechanism than is used for excluding just external headers.
target_statement = f"filter('^(//|@//)',{target_statement})"
compile_commands.extend(_get_compile_commands_for_aquery(target_statement, additional_flags))
if not compile_commands:
log_warning(f""">>> Bazel lists no applicable compile commands for {target}
target_statement_candidates.append(f"filter('^(//|@//)',{target_statement})")

found = False
for target_statement in target_statement_candidates:
commands = list(_get_compile_commands_for_aquery(target_statement, additional_flags, file_path))
compile_commands.extend(commands) # If we did the work to generate a command, we'll update it, whether it's for the requested file or not.
if file_flags:
if any(command['file'].endswith(file_path) for command in commands):
found = True
break
log_info(f""">>> Couldn't quickly find a compile command for {file_path} in {target} under {target_statement}
Continuing gracefully...""")

if file_flags and not found:
log_warning(f""">>> Couldn't quickly find a compile command for {file_path} in {target}
Continuing gracefully...""")

if not compile_commands:
log_warning(f""">>> Bazel lists no applicable compile commands for {target}
If this is a header-only library, please instead specify a test or binary target that compiles it (search "header-only" in README.md).
Continuing gracefully...""") # TODO simplify/unify logic around warning about not being able to find commands.
Continuing gracefully...""")

log_success(f">>> Finished extracting commands for {target}")
return compile_commands
Expand Down