The Datadog Native Profiler for Linux.

ddprof is a command-line utility to gather profiling data. After install you will continuously see where your application is spending CPU and memory allocations. The data will be available in the /profiling section of the Datadog UI.

Our official documentation is available here. Our pre-built binaries are compatible with both musl and glibc. You should not need to recompile ddprof from source.

An installation guide is available here. Check out our Release page for our latest release. Download the release and extract ddprof. Instrumenting your application should be as simple as adding ddprof in front of your usual command line.

To install the profiler, check out our installation-helpers bellow.

The following command will run ddprof with the default settings (CPU and allocations)

ddprof -S service-name-for-my-program ./my_program arg1 arg2

Profiling data shows up in the /profiling section of your Datadog UI. Specifying a service name will help you select your profiling data. Refer to commands for a more advanced usage of ddprof.

Checkout our build section here.

The target machine must have perf_event_paranoid set to 2 or lower.

# needs to be less than or equal to 2
cat /proc/sys/kernel/perf_event_paranoid

Here is an example adding a startup configuration to your system. This requires a system restart.

sudo sh -c 'echo kernel.perf_event_paranoid=2 > /etc/sysctl.d/perf_event_paranoid_2.conf'

Alternatively you can use CAP_SYS_ADMIN or sudo as a one off test mechanism, more in the Troubleshooting section. Don't hesitate to reach-out if you are not able to use our profiler!

It is recommended to have an agent setup on the system you are profiling. By default the profiler will target localhost:8126 (the default trace agent endpoint). The DD_TRACE_AGENT_URL environment variable can be used to override this setting.

Install curl.

sudo apt-get update && \
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y curl

Download the latest ddprof version, installing it to /usr/local/bin.

# ARCH should hold amd64 or arm64 
ARCH=$(dpkg --print-architecture) && \
url_release="https://github.com/DataDog/ddprof/releases/latest/download/ddprof-${ARCH}" && \
curl -L -o ddprof ${url_release} && \
chmod 755 ddprof && \
sudo mv ddprof /usr/local/bin && \
ddprof --version

If your application runs with the default allocator (or a dynamically linked jemalloc), we will instrument your allocations. Launch this command and checkout the live heap profiles in the UI. You can use the compare tool to understand allocations which are growing over time.

ddprof -S my-service-name --preset cpu_live_heap -l warn --worker-period 15000 ./my-application

⚠️ The --worker-period flag is used to avoid the resets of the heap profile. The default value is 4 hours. This changes to 15000 minutes (10+ days).

You can instrument all running applications with the following command. You can use a fake service name to find your profiles.

sudo ddprof -S host-123 -g -l warn

ddprof is a wrapper, so using it should be as simple as injecting the binary into your container and wrapping your entrypoint. ddprof will use environment variables if they are available, overriding them with commandline parameters if given.

• Minimal interference to execution of instrumented processes
• ddprof's Memory usage is sandboxed

• By working in user space, ddprof can instrument allocations with low overhead

Architectural showpieces and such will always be available in the docs/ folder.

• Build
• Design
• Automatically updated list of commads
• Troubleshooting

Any contribution is welcome. Please check out our guidelines here.