Initial perf jitdump implementation #26897

sdmaclea · 2019-09-25T21:08:55Z

This initial commit:

creates the jit-%d.dump file
writes the file header
creates the mmap mark
adds JIT_CODE_LOAD records for JIT generated code and for stubs.

It has been lightly tested on Ubuntu 16.04 with perf built from Ubuntu-azure-4.15.0-1057.62 with the patch https://lkml.org/lkml/2019/9/20/862

export COMPlus_PerfMapEnabled=1
~/git/linux-azure-xenial/tools/perf/perf record -k 1 -g /home/stmaclea/git/coreclr/bin/tests/Linux.x64.Debug/Tests/Core_Root/corerun helloworld.dll
~/git/linux-azure-xenial/tools/perf/perf --debug verbose=10 inject --input perf.data --jit --output perfjit.data
~/git/linux-azure-xenial/tools/perf/perf report --input perfjit.data

Basic functionality seems correct, however there are sample which are hitting in perf-*.map, but not found in jit-*.dump.

It needs to be further debugged.

Fixes #26842

/cc @dotnet/dotnet-diag

sdmaclea · 2019-09-25T23:29:32Z

I looked at the missing symbols. It turns out when we grow the code page, ~~we remap the entire page. This obscures the previous JIT_CODE_LOAD records. Emitting a JIT_CODE_MOVE for all previous JIT_CODE_LOAD records will likely solve this problem.~~

EDIT:

When we grow the code pages, the kernel is coalescing the pages and sending an anonymous mmap event for the coalesced region.

src/pal/src/misc/perfjitdump.cpp

src/inc/perfjitdump.h

noahfalk

Looks fine so far, aside from the stuff Jan already noted

src/vm/perfmap.cpp

Writer should always write the same value, independent of endianness.

sdmaclea · 2019-10-03T19:14:36Z

Status:

The draft functional implementation is complete.

Testing with perf built from:

Repo: git:https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 
Branch: perf/core
Patches:
- https://lkml.org/lkml/2019/9/20/862
- https://lkml.org/lkml/2019/10/2/875

cd tools/perf && make DEBUG=1

I captured a fairly simple app that printed hello world + a counter 100K times.

I was able to see and annotate R2R and JIT generated method (disassembly), some for multiple compilation tiers.

The vast majority of symbols were resolved. There were a handful of unknown addresses. I think we have missed logging of one type of stub.

The call stacks looked believable and complete. Unwindinfo didn't seem critically important for 'fp' based perf callstacks. They are probably needed for perf dwarf call stack support.

Future

Add unwind info.
Add debug info.

Revise PAL API Add error handling Clean up #ifdef CROSSGEN Add COMPlus_PerfMapJitDumpPath Add placeholder debuginfo and unwindinfo arguments

sdmaclea · 2019-10-04T18:54:54Z

I have addressed all feedback and finished my to do list. This should be ready for review.

/cc @brianrob

src/inc/clrconfigvalues.h

src/pal/src/misc/perfjitdump.cpp

janvorli · 2019-10-04T19:30:08Z

src/inc/perfjitdump.h

- // Finish the jitdump file
- static void Finish();
-};
+int


These should go to pal.h where all other PAL function declarations are located.

src/pal/src/misc/perfjitdump.cpp

src/vm/perfmap.cpp

janvorli · 2019-10-04T20:46:52Z

src/vm/perfmap.cpp

@@ -274,6 +281,11 @@ void PerfMap::LogPreCompiledMethod(MethodDesc * pMethod, PCODE pCode)
 {
 LIMITED_METHOD_CONTRACT;

+ if (s_Current == nullptr)


It seems that if one of these logging functions races with the PerfMap::Destroy, we have a potential to crash in the code below.

This doesn't look like an issue with this function (since it is just an early out), but for other preexisting functions it is potentially an issue.

I'll open a separate PR to fix this.

Revise error handling Remove perfjitdump.h Fix #if CROSSGEN_COMPILE

While a JIT is jitting code it will eventually need to commit more pages and change these pages to executable permissions. Typically the JIT will want these colocated to minimize branch displacements. The kernel will coalesce these anonymous mapping with identical permissions before sending an MMAP event for the new pages. This means the mmap event for the new pages will include the older pages. These anonymous mmap events will obscure the jitdump injected pseudo events. This means that the jitdump generated symbols, machine code, debugging info, and unwind info will no longer be used. Observations: When a process emits a jit dump marker and a jitdump file, the perf-xxx.map file represents inferior information which has been superceded by the jitdump jit-xxx.dump file. Further the '//anon*' mmap events are only required for the legacy perf-xxx.map mapping. When attaching to an existing process, the synthetic anon map events are given a time stamp of -1. These should not obscure the jitdump events which have an actual time. Summary: Use thread->priv to store whether a jitdump file has been processed During "perf inject --jit", discard "//anon*" mmap events for any pid which has sucessfully processed a jitdump file. Committer testing: // jitdump case perf record <app with jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events removed perf script --input perfjit.data --show-mmap-events | grep '//anon' // no jitdump case perf record <app without jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events not removed perf script --input perfjit.data --show-mmap-events | grep '//anon' Repro: This issue was discovered while testing the initial CoreCLR jitdump implementation. dotnet/coreclr#26897. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Arnaldo Carvalho de Melo <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Alexander Shishkin <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Stephane Eranian <[email protected]> Cc: [email protected] Signed-off-by: Steve MacLean <[email protected]>

While a JIT is jitting code it will eventually need to commit more pages and change these pages to executable permissions. Typically the JIT will want these colocated to minimize branch displacements. The kernel will coalesce these anonymous mapping with identical permissions before sending an MMAP event for the new pages. This means the mmap event for the new pages will include the older pages. These anonymous mmap events will obscure the jitdump injected pseudo events. This means that the jitdump generated symbols, machine code, debugging info, and unwind info will no longer be used. Observations: When a process emits a jit dump marker and a jitdump file, the perf-xxx.map file represents inferior information which has been superceded by the jitdump jit-xxx.dump file. Further the '//anon*' mmap events are only required for the legacy perf-xxx.map mapping. When attaching to an existing process, the synthetic anon map events are given a time stamp of -1. These should not obscure the jitdump events which have an actual time. Summary: Use thread->priv to store whether a jitdump file has been processed During "perf inject --jit", discard "//anon*" mmap events for any pid which has sucessfully processed a jitdump file. Committer testing: // jitdump case perf record <app with jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events removed perf script --input perfjit.data --show-mmap-events | grep '//anon' // no jitdump case perf record <app without jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events not removed perf script --input perfjit.data --show-mmap-events | grep '//anon' Repro: This issue was discovered while testing the initial CoreCLR jitdump implementation. dotnet/coreclr#26897. Signed-off-by: Steve MacLean <[email protected]>

**perf-<pid>.map and jit-<pid>.dump designs: When a JIT generates code to be executed, it must allocate memory and mark it executable using an mmap call. *** perf-<pid>.map design The perf-<pid>.map assumes that any sample recorded in an anonymous memory page is JIT code. It then tries to resolve the symbol name by looking at the process' perf-<pid>.map. *** jit-<pid>.dump design The jit-<pid>.dump mechanism takes a different approach. It requires a JIT to write a `<path>/jit-<pid>.dump` file. This file must also be mmapped so that perf inject -jit can find the file. The JIT must also add JIT_CODE_LOAD records for any functions it generates. The records are timestamped using a clock which can be correlated to the perf record clock. After perf record, the `perf inject -jit` pass parses the recording looking for a `<path>/jit-<pid>.dump` file. When it finds the file, it parses it and for each JIT_CODE_LOAD record: * creates an elf file `<path>/jitted-<pid>-<code_index>.so * injects a new mmap record mapping the new elf file into the process. *** Coexistence design The kernel and perf support both of these mechanisms. We need to make sure perf works on an app supporting either or both of these mechanisms. Both designs rely on mmap records to determine how to resolve an ip address. The mmap records of both techniques by definition overlap. When the JIT compiles a method, it must: * allocate memory (mmap) * add execution privilege (mprotect or mmap. either will generate an mmap event form the kernel to perf) * compile code into memory * add a function record to perf-<pid>.map and/or jit-<pid>.dump Because the jit-<pid>.dump mechanism supports greater capabilities, perf prefers the symbols from jit-<pid>.dump. It implements this based on timestamp ordering of events. There is an implicit ASSUMPTION that the JIT_CODE_LOAD record timestamp will be after the // anon mmap event that was generated during memory allocation or adding the execution privilege setting. *** Problems with the ASSUMPTION The ASSUMPTION made in the Coexistence design section above is violated in the following scenario. *** Scenario While a JIT is jitting code it will eventually need to commit more pages and change these pages to executable permissions. Typically the JIT will want these collocated to minimize branch displacements. The kernel will coalesce these anonymous mapping with identical permissions before sending an MMAP event for the new pages. The address range of the new mmap will not be just the most recently mmap pages. It will include the entire coalesced mmap region. See mm/mmap.c unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf) { ... /* * Can we just expand an old mapping? */ ... perf_event_mmap(vma); ... } *** Symptoms The coalesced // anon mmap event will be timestamped after the JIT_CODE_LOAD records. This means it will be used as the most recent mapping for that entire address range. For remaining events it will look at the inferior perf-<pid>.map for symbols. If both mechanisms are supported, the symbol will appear twice with different module names. This causes weird behavior in reporting. If only jit-<pid>.dump is supported, the symbol will no longer be resolved. ** Implemented solution This patch solves the issue by removing // anon mmap events for any process which has a valid jit-<pid>.dump file. It tracks on a per process basis to handle the case where some running apps support jit-<pid>.dump, but some only support perf-<pid>.map. It adds new assumptions: * // anon mmap events are only required for perf-<pid>.map support. * An app that uses jit-<pid>.dump, no longer needs perf-<pid>.map support. It assumes that any perf-<pid>.map info is inferior. *** Details Use thread->priv to store whether a jitdump file has been processed During "perf inject --jit", discard "//anon*" mmap events for any pid which has sucessfully processed a jitdump file. ** Committer testing: // jitdump case perf record <app with jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events removed perf script --input perfjit.data --show-mmap-events | grep '//anon' // no jitdump case perf record <app without jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events not removed perf script --input perfjit.data --show-mmap-events | grep '//anon' ** Repro: This issue was discovered while testing the initial CoreCLR jitdump implementation. dotnet/coreclr#26897. ** Alternate solutions considered These were also briefly considered * Change kernel to not coalesce mmap regions. * Change kernel reporting of coalesced mmap regions to perf. Only include newly mapped memory. * Only strip parts of // anon mmap events overlapping existing jitted-<pid>-<code_index>.so mmap events. Cc: Peter Zijlstra <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Arnaldo Carvalho de Melo <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Alexander Shishkin <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Stephane Eranian <[email protected]> Cc: [email protected] Signed-off-by: Steve MacLean <[email protected]>

**perf-<pid>.map and jit-<pid>.dump designs: When a JIT generates code to be executed, it must allocate memory and mark it executable using an mmap call. *** perf-<pid>.map design The perf-<pid>.map assumes that any sample recorded in an anonymous memory page is JIT code. It then tries to resolve the symbol name by looking at the process' perf-<pid>.map. *** jit-<pid>.dump design The jit-<pid>.dump mechanism takes a different approach. It requires a JIT to write a `<path>/jit-<pid>.dump` file. This file must also be mmapped so that perf inject -jit can find the file. The JIT must also add JIT_CODE_LOAD records for any functions it generates. The records are timestamped using a clock which can be correlated to the perf record clock. After perf record, the `perf inject -jit` pass parses the recording looking for a `<path>/jit-<pid>.dump` file. When it finds the file, it parses it and for each JIT_CODE_LOAD record: * creates an elf file `<path>/jitted-<pid>-<code_index>.so * injects a new mmap record mapping the new elf file into the process. *** Coexistence design The kernel and perf support both of these mechanisms. We need to make sure perf works on an app supporting either or both of these mechanisms. Both designs rely on mmap records to determine how to resolve an ip address. The mmap records of both techniques by definition overlap. When the JIT compiles a method, it must: * allocate memory (mmap) * add execution privilege (mprotect or mmap. either will generate an mmap event form the kernel to perf) * compile code into memory * add a function record to perf-<pid>.map and/or jit-<pid>.dump Because the jit-<pid>.dump mechanism supports greater capabilities, perf prefers the symbols from jit-<pid>.dump. It implements this based on timestamp ordering of events. There is an implicit ASSUMPTION that the JIT_CODE_LOAD record timestamp will be after the // anon mmap event that was generated during memory allocation or adding the execution privilege setting. *** Problems with the ASSUMPTION The ASSUMPTION made in the Coexistence design section above is violated in the following scenario. *** Scenario While a JIT is jitting code it will eventually need to commit more pages and change these pages to executable permissions. Typically the JIT will want these collocated to minimize branch displacements. The kernel will coalesce these anonymous mapping with identical permissions before sending an MMAP event for the new pages. The address range of the new mmap will not be just the most recently mmap pages. It will include the entire coalesced mmap region. See mm/mmap.c unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf) { ... /* * Can we just expand an old mapping? */ ... perf_event_mmap(vma); ... } *** Symptoms The coalesced // anon mmap event will be timestamped after the JIT_CODE_LOAD records. This means it will be used as the most recent mapping for that entire address range. For remaining events it will look at the inferior perf-<pid>.map for symbols. If both mechanisms are supported, the symbol will appear twice with different module names. This causes weird behavior in reporting. If only jit-<pid>.dump is supported, the symbol will no longer be resolved. ** Implemented solution This patch solves the issue by removing // anon mmap events for any process which has a valid jit-<pid>.dump file. It tracks on a per process basis to handle the case where some running apps support jit-<pid>.dump, but some only support perf-<pid>.map. It adds new assumptions: * // anon mmap events are only required for perf-<pid>.map support. * An app that uses jit-<pid>.dump, no longer needs perf-<pid>.map support. It assumes that any perf-<pid>.map info is inferior. *** Details Use thread->priv to store whether a jitdump file has been processed During "perf inject --jit", discard "//anon*" mmap events for any pid which has sucessfully processed a jitdump file. ** Testing: // jitdump case perf record <app with jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events removed perf script --input perfjit.data --show-mmap-events | grep '//anon' // no jitdump case perf record <app without jitdump> perf inject --jit --input perf.data --output perfjit.data // verify mmap "//anon" events present initially perf script --input perf.data --show-mmap-events | grep '//anon' // verify mmap "//anon" events not removed perf script --input perfjit.data --show-mmap-events | grep '//anon' ** Repro: This issue was discovered while testing the initial CoreCLR jitdump implementation. dotnet/coreclr#26897. ** Alternate solutions considered These were also briefly considered: * Change kernel to not coalesce mmap regions. * Change kernel reporting of coalesced mmap regions to perf. Only include newly mapped memory. * Only strip parts of // anon mmap events overlapping existing jitted-<pid>-<code_index>.so mmap events. Signed-off-by: Steve MacLean <[email protected]> Acked-by: Ian Rogers <[email protected]> Cc: Alexander Shishkin <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Stephane Eranian <[email protected]> Link: http:https://lore.kernel.org/lkml/1590544271-125795-1-git-send-email-steve.maclean@linux.microsoft.com Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

AndyAyersMS · 2020-09-11T02:31:37Z

@sdmaclea does this work out of the box or do I need some special build?

sdmaclea · 2020-09-11T16:32:11Z

To fix all known issues, you should use a build of perf from the linux tip.

The last known issue was fixed with torvalds/linux@c8f6ae1

Building perf is not that hard.

Install kernel required tool chains (See for instance https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel Build Environment section)
cd tools\perf
make

brianrob · 2020-09-11T17:24:21Z

Also worth noting that perfcollect now has support for using this functionality by default. FWIW, I have had reasonable luck using versions of perf that are available with most recent distros, but if you are running into issues, building the tip is a good way to ensure that they are not known issues that @sdmaclea has already fixed.

AndyAyersMS · 2020-09-11T19:36:43Z

Ok, thanks... will give this a try.

AndyAyersMS · 2020-09-16T18:48:46Z

Confessing my ignorance here. What can I do If I'm stuck on some older kernel version ( say 4.10)... does building the latest perf from there work? I end up with perf version 4.4.232.

I also realized I needed to run perf inject to pick up the jit events.

Now I can sometimes see jitted code, eg one of the five methods below:

+   90.36%     0.00%  corerun          libcoreclr.so               [.] RunMain                                                                                                                                                              ▒
+   89.63%     0.02%  corerun          perf-36392.map              [.] void [r41741] X::Main()[Optimized]                                                                                                                                   ▒
+   64.42%     3.16%  corerun          perf-36392.map              [.] void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::IntroSort(valuetype System.Span`1<!0>,int32)[OptimizedTier1]        ▒
+   63.75%     0.05%  corerun          jitted-36392-1185.so        [.] instance void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::Sort(valuetype System.Span`1<!0>,class System.Collections.G▒
+   33.08%    33.06%  corerun          perf-36392.map              [.] void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::InsertionSort(valuetype System.Span`1<!0>)[OptimizedTier1]          ▒
+   27.36%    27.35%  corerun          perf-36392.map              [.] int32 [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::PickPivotAndPartition(valuetype System.Span`1<!0>)[OptimizedTier1] ▒
+   16.63%     0.00%  corerun          libcoreclr.so               [.] MethodDesc::DoPrestub

Do the jit events this play well with the tiering annotations? I notice above you were running debug so didn't see tiering.

Also COMPlus_PerfMapShowOptimizationTiers=0 does not seem to suppress these annotations.

brianrob · 2020-09-21T23:45:40Z

Yes, building a new perf and using it with an older kernel should be just fine. For the most part, the user-mode tool is backwards compatible, and I've been told forwards compatible...

Yes, @kouvel did some work a while back to add tiering labels to the symbols so they should show up just fine in this view. In PerfView they are stripped but can be enabled by looking at an optimization tiers view (you'll see it in the view name) or running PerfView with /ShowOptimizationTiers to enable it for all stack views.

AndyAyersMS · 2020-09-22T20:24:54Z

Thanks all. Building the latest seems to be working:

+   89.74%     0.01%  corerun          jitted-44217-820.so   [.] void [r41741] X::Main()
+   64.11%     3.21%  corerun          jitted-44217-1150.so  [.] void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::IntroSort(valuetype System.Span`1<!0>,int32)
+   63.38%     0.03%  corerun          jitted-44217-1185.so  [.] instance void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::Sort(valuetype System.Span`1<!0>,class System.Collections.Generic.
+   33.08%    33.08%  corerun          jitted-44217-1165.so  [.] void [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::InsertionSort(valuetype System.Span`1<!0>)
+   27.14%    27.14%  corerun          jitted-44217-1164.so  [.] int32 [System.Private.CoreLib] System.Collections.Generic.GenericArraySortHelper`1[System.Int32]::PickPivotAndPartition(valuetype System.Span`1<!0>)
+   16.78%     0.01%  corerun          libcoreclr.so         [.] MethodDesc::DoPrestub

sdmaclea added os-linux Linux OS (any supported distro) area-Diagnostics labels Sep 25, 2019

sdmaclea added this to the 5.0 milestone Sep 25, 2019

sdmaclea requested review from jkotas and janvorli September 25, 2019 21:08

sdmaclea self-assigned this Sep 25, 2019

janvorli reviewed Sep 27, 2019

View reviewed changes

src/pal/src/misc/perfjitdump.cpp Outdated Show resolved Hide resolved

src/inc/perfjitdump.h Outdated Show resolved Hide resolved

noahfalk reviewed Sep 30, 2019

View reviewed changes

src/vm/perfmap.cpp Show resolved Hide resolved

sdmaclea added 4 commits October 3, 2019 14:23

Initial perf jitdump implementation

3b0148c

Fix spelling

1793bad

Correct JIT_DUMP_MAGIC

9bd19d0

Writer should always write the same value, independent of endianness.

Add PreJIT to jitdump

2c25dff

sdmaclea force-pushed the jitdump branch from dd688b2 to 2c25dff Compare October 3, 2019 18:52

Feedback

4c69ef8

Revise PAL API Add error handling Clean up #ifdef CROSSGEN Add COMPlus_PerfMapJitDumpPath Add placeholder debuginfo and unwindinfo arguments

sdmaclea changed the title ~~WIP Initial perf jitdump implementation~~ Initial perf jitdump implementation Oct 4, 2019

jkotas reviewed Oct 4, 2019

View reviewed changes

src/inc/clrconfigvalues.h Outdated Show resolved Hide resolved

RETAIL ... PerfMapJitDumpPath

70c9724

janvorli reviewed Oct 4, 2019

View reviewed changes

Feedback

6feed06

Revise error handling Remove perfjitdump.h Fix #if CROSSGEN_COMPILE

sdmaclea merged commit 0868d20 into dotnet:master Oct 7, 2019

This was referenced Jan 31, 2020

FEATURE_PERFMAP missing stubs? dotnet/runtime#13542

Closed

PerfMap race in destroy dotnet/runtime#13543

Closed

jkotas mentioned this pull request Apr 27, 2020

Resolving framework symbols dotnet/source-build#1564

Closed

AndyAyersMS mentioned this pull request Sep 14, 2020

[ARM64] Performance regression: Sorting arrays of primitive types dotnet/runtime#41741

Closed

vchuravy mentioned this pull request Sep 21, 2020

Enable Perf JITEvents by default JuliaLang/julia#37682

Merged

sdmaclea deleted the jitdump branch September 22, 2020 21:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial perf jitdump implementation #26897

Initial perf jitdump implementation #26897

sdmaclea commented Sep 25, 2019 •

edited

Loading

sdmaclea commented Sep 25, 2019 •

edited

Loading

noahfalk left a comment

sdmaclea commented Oct 3, 2019 •

edited

Loading

sdmaclea commented Oct 4, 2019

janvorli Oct 4, 2019

janvorli Oct 4, 2019

sdmaclea Oct 4, 2019

AndyAyersMS commented Sep 11, 2020

sdmaclea commented Sep 11, 2020 •

edited

Loading

brianrob commented Sep 11, 2020

AndyAyersMS commented Sep 11, 2020

AndyAyersMS commented Sep 16, 2020

brianrob commented Sep 21, 2020

AndyAyersMS commented Sep 22, 2020

Initial perf jitdump implementation #26897

Initial perf jitdump implementation #26897

Conversation

sdmaclea commented Sep 25, 2019 • edited Loading

sdmaclea commented Sep 25, 2019 • edited Loading

noahfalk left a comment

Choose a reason for hiding this comment

sdmaclea commented Oct 3, 2019 • edited Loading

sdmaclea commented Oct 4, 2019

janvorli Oct 4, 2019

Choose a reason for hiding this comment

janvorli Oct 4, 2019

Choose a reason for hiding this comment

sdmaclea Oct 4, 2019

Choose a reason for hiding this comment

AndyAyersMS commented Sep 11, 2020

sdmaclea commented Sep 11, 2020 • edited Loading

brianrob commented Sep 11, 2020

AndyAyersMS commented Sep 11, 2020

AndyAyersMS commented Sep 16, 2020

brianrob commented Sep 21, 2020

AndyAyersMS commented Sep 22, 2020

sdmaclea commented Sep 25, 2019 •

edited

Loading

sdmaclea commented Sep 25, 2019 •

edited

Loading

sdmaclea commented Oct 3, 2019 •

edited

Loading

sdmaclea commented Sep 11, 2020 •

edited

Loading