Python Memory Profiling with tracemalloc

What is tracemalloc?

Available since: Python 3.4 (PEP 454)

Tracemalloc is a powerful debugging tool included in Python's standard library that allows developers to trace memory blocks allocated by Python programs. Unlike traditional memory profilers that only show overall memory usage, tracemalloc provides detailed insights into where and how memory is being allocated in your code.

With tracemalloc, you can:

  • Track where objects are created in your code
  • Analyze memory usage patterns
  • Compare memory snapshots to identify leaks
  • View statistics on memory blocks by file and line number

Installation & Setup

Since tracemalloc is part of the Python standard library, no additional installation is required. However, to use it effectively, you need to enable it at the start of your program.

Python
import tracemalloc

# Start tracing Python memory allocations
tracemalloc.start()

# Your code here
# ...

# Get the current snapshot
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

# Print statistics
for stat in top_stats[:10]:
    print(stat)
Environment Setup
# Set environment variable before running your script
# Linux/macOS
export PYTHONTRACEMALLOC=1

# Windows
set PYTHONTRACEMALLOC=1

# Then run your Python program
python your_script.py
Advanced Setup
import tracemalloc

# Start tracemalloc with custom parameters
# The parameter 25 is the maximum number of frames 
# to store in a traceback
tracemalloc.start(25)

# Get current size and peak
current, peak = tracemalloc.get_traced_memory()
print(f"Current memory usage: {current / 1024}KB")
print(f"Peak memory usage: {peak / 1024}KB")

# Reset peak to current
tracemalloc.reset_peak()

# Stop tracking when done
tracemalloc.stop()

Basic Usage

The workflow for using tracemalloc typically follows these steps:

  1. 1

    Start Tracing

    Enable the tracemalloc module at the beginning of your program.

  2. 2

    Run Your Code

    Execute the code you want to analyze for memory usage.

  3. 3

    Take Snapshots

    Capture the memory state at specific points in execution.

  4. 4

    Analyze Results

    Review statistics or compare snapshots to identify issues.

Common Use Cases

  • Finding memory leaks - Identify objects that aren't being properly garbage collected
  • Optimizing memory usage - Discover which parts of your code consume the most memory
  • Debugging unexpected memory growth - Track down where memory is being allocated over time
  • Profiling memory-intensive applications - Understand memory patterns in data processing pipelines

Key Features

Memory Allocation Tracking

Trace where objects are allocated with detailed tracebacks

  • Pinpoint exact file and line numbers
  • Track objects by size and count
  • Filter by specific modules or files

Snapshot Capabilities

Capture memory states for analysis and comparison

  • Take snapshots at different execution points
  • Compare snapshots to detect memory changes
  • Filter by specific criteria for focused analysis

Memory Usage Statistics

Get comprehensive memory usage data

  • Monitor current and peak memory usage
  • View statistics by file, line, or category
  • Track memory blocks by size distributions

Debugging Utilities

Tools to help interpret memory data

  • Format tracebacks similar to standard Python tracebacks
  • Filter noise from system libraries
  • Integrate with other debugging tools

Working with Snapshots

Snapshots are a core concept in tracemalloc that allow you to capture the memory state at a specific point in time. By comparing snapshots taken at different times, you can identify memory leaks and understand memory usage patterns.

Taking and Comparing Snapshots
import tracemalloc

# Start tracing memory
tracemalloc.start()

# ... some initial code ...

# Take a first snapshot
snapshot1 = tracemalloc.take_snapshot()

# ... more code that might cause memory changes ...

# Take a second snapshot
snapshot2 = tracemalloc.take_snapshot()

# Compare snapshots
top_stats = snapshot2.compare_to(snapshot1, 'lineno')

print("Memory usage growth:")
for stat in top_stats[:10]:
    print(stat)

Pro Tip

When comparing snapshots, focus on the biggest differences first. These are often the most likely candidates for memory leaks or optimization opportunities.

Memory Statistics

Tracemalloc provides detailed statistics about memory usage in your application, helping you understand where memory is being allocated and how it's being used.

Available Statistics Methods

Method Description
statistics('lineno') Group by source file and line number
statistics('filename') Group by source file
statistics('traceback') Group by traceback
get_traced_memory() Get current and peak memory usage
get_tracemalloc_memory() Memory used by tracemalloc itself

Code Examples

Finding Memory Leaks

Python
import tracemalloc
import gc

def detect_leaks():
    tracemalloc.start()
    
    # Run a baseline collection
    gc.collect()
    
    # Record the starting state
    snapshot1 = tracemalloc.take_snapshot()
    
    # Run the code that might leak memory
    potentially_leaky_function()
    
    # Force garbage collection to remove objects
    # that are no longer referenced
    gc.collect()
    
    # Take a second snapshot
    snapshot2 = tracemalloc.take_snapshot()
    
    # Compare snapshots
    top_stats = snapshot2.compare_to(snapshot1, 'lineno')
    
    print("Potential memory leaks:")
    for stat in top_stats[:5]:
        print(stat)

def potentially_leaky_function():
    # This is just a placeholder for your actual code
    large_list = [0] * 1000000  # Allocate a large list
    
    # In a real leak, this object would be
    # accidentally kept alive somewhere

detect_leaks()

Memory Usage Timeline

Python
import tracemalloc
import time

# Start tracking memory
tracemalloc.start()
snapshots = []

# Function to record a memory snapshot with timestamp
def record_memory():
    snapshot = tracemalloc.take_snapshot()
    current, peak = tracemalloc.get_traced_memory()
    snapshots.append((time.time(), current, snapshot))
    return current

# Record initial state
record_memory()
print("Initial memory:", snapshots[0][1] / 1024, "KB")

# Record several points for analysis
for i in range(3):
    # Simulate work that uses memory
    data = [object() for _ in range(100000)]
    
    # Record the memory at this point
    current = record_memory()
    print(f"Memory after step {i+1}:", current / 1024, "KB")
    
    # For the last iteration, keep the reference
    # to prevent garbage collection
    if i < 2:
        del data  # Remove reference to allow cleanup

# Print memory growth timeline
print("\nMemory Timeline:")
for i, (timestamp, memory, _) in enumerate(snapshots):
    if i > 0:
        prev_time, prev_mem = snapshots[i-1][0], snapshots[i-1][1]
        time_diff = timestamp - prev_time
        mem_diff = memory - prev_mem
        print(f"Step {i}: {memory/1024:.2f} KB " + 
              f"({mem_diff/1024:+.2f} KB in {time_diff:.2f}s)")

Resources

Official

Python Documentation

The comprehensive official documentation for the tracemalloc module in Python 3.13.3, updated April 2025.

Read More
Tutorial

Memory Profiling Guide

A practical guide to memory profiling in Python using the standard library's tracemalloc module.

Read More
Advanced

Fixing Memory Leaks

An in-depth guide on using tracemalloc for memory profiling to diagnose and fix memory leaks in Python applications.

Read More
Video

Memory Profiling in Python

A comprehensive video tutorial on understanding and fixing memory issues in Python using tracemalloc and other tools.

Watch

Frequently Asked Questions

How much overhead does tracemalloc add?

Tracemalloc adds some memory and CPU overhead to your application. The memory overhead depends on the number of frames tracked per allocation and the number of tracked allocations. In general, the overhead ranges from 5% to 20% depending on your application.

Use tracemalloc.get_tracemalloc_memory() to see how much memory tracemalloc itself is using.

Can tracemalloc track C extension memory usage?

No, tracemalloc only tracks memory blocks allocated by Python itself. Memory allocated by C extensions (like NumPy arrays using their own allocators) is not tracked unless the extension explicitly uses Python's memory allocation APIs.

For tracking memory in C extensions, consider external tools like Valgrind or Memory Profiler.

How can I reduce the amount of traceback data collected?

You can control the amount of traceback data collected in two ways:

  1. Limit the number of frames when starting tracemalloc: tracemalloc.start(nframe=1)
  2. Filter traces when taking a snapshot: snapshot.filter_traces(filters)

Both approaches can significantly reduce memory overhead for large applications.