gzhttp: Add BREACH mitigation

[klauspost]

See #761

BREACH mitigation

BREACH is a specialized attack where attacker controlled data is injected alongside secret data in a response body. This can lead to sidechannel attacks, where observing the compressed response size can reveal if there are overlaps between the secret data and the injected data.

For more information see https://breachattack.com/

It can be hard to judge if you are vulnerable to BREACH. In general, if you do not include any user provided content in the response body you are safe, but if you do, or you are in doubt, you can apply mitigations.

gzhttp can apply Heal the Breach, or improved content aware padding.

// RandomJitter adds 1->n random bytes to output based on checksum of payload.
// Specify the amount of input to buffer before applying jitter.
// This should cover the sensitive part of your response.
// This can be used to obfuscate the exact compressed size.
// Specifying 0 will use a buffer size of 64KB.
// If a negative buffer is given, the amount of jitter will not be content dependent.
// This provides *less* security than applying content based jitter.
func RandomJitter(n, buffer int) option {
...

The jitter is added as a "Comment" field. This field has a 1 byte overhead, so actual extra size will be 2 -> n+1 (inclusive).

A good option would be to apply 32 random bytes, with default 64KB buffer: gzhttp.RandomJitter(32, 0).

Note that flushing the data forces the padding to be applied, which means that only data before the flush is considered for content aware padding.

Examples

Adding the option gzhttp.RandomJitter(32, 50000) will apply from 1 up to 32 bytes of random data to the output.

The number of bytes added depends on the content of the first 50000 bytes, or all of them if the output was less than that.

Adding the option gzhttp.RandomJitter(32, -1) will apply from 1 up to 32 bytes of random data to the output. Each call will apply a random amount of jitter. This should be considered less secure than content based jitter.

This can be used if responses are very big, deterministic and the buffer size would be too big to cover where the mutation occurs.

[d-z-m]

Adding the option gzhttp.RandomJitter(32, 50000) will apply from 1 up to 32 bytes of random data to the output.

The padding isn't random, correct? It is derived from the c.randomJitter = bytes.Repeat([]byte("Padding-"), 1+(n/8)) buffer. While the length of the padding is the little-endian Uint32 representation of the first 4 bytes of the SHA256 checksum of the page.

[klauspost]

Correct.

The padding in the comment is Padding-Padding-Padding-Padding-Pad.....

The length is 1 + sha256(payload) MOD max_length or just random from crypto/rand if buffer < 0.

[d-z-m]

Correct.

The padding in the comment is Padding-Padding-Padding-Padding-Pad.....

The length is 1 + sha256(payload) MOD max_length

Gotcha, ideally the gzhttp/README.md would reflect this, or say something equivalent. Otherwise LGTM 👍

[greatroar]

Is this really necessary? Not Heal the Breach I mean, but the random jitter? If I understand the paper correctly, HTB only needs a filename or comment field of random length, not random content, because neither of those fields affects the compression of the content. See, e.g., the Django implementation.

[klauspost]

@greatroar The content is not random.

The padding in the comment is Padding-Padding-Padding-Padding-Pad.....

Literally.

(and it uses the comment, not the file name)

[greatroar]

Sorry, I still misunderstood what you said. Never mind.

[klauspost]

@greatroar With a random padding you can still easily deduce the compressed size. With 100 bytes padding:

package main

import (
	"bytes"
	"compress/gzip"
	"crypto/rand"
	"fmt"
	"math"
)

func main() {
	payload := `Error: msgp: wanted array of size 7; got 5 at Cache/testbucket/go3/src/runtime/mranges_test.go (msgp.ArrayError)
       5: e:\minio\minio\internal\logger\logger.go:258:logger.LogIf()
       4: e:\minio\minio\internal\logger\logonce.go:104:logger.(*logOnceType).logOnceIf()
       3: e:\minio\minio\internal\logger\logonce.go:135:logger.LogOnceIf()
       2: e:\minio\minio\cmd\data-usage-cache.go:903:cmd.(*dataUsageCache).load()
       1: e:\minio\minio\cmd\erasure.go:467:cmd.erasureObjects.nsScanner.func3()`

	const maxDelta = 99
	tries := 0
	var buf bytes.Buffer
	gw := gzip.NewWriter(&buf)
	gw.Write([]byte(payload))
	gw.Close()
	want := buf.Len()
	var minSeen, maxSeen = math.MaxInt32, 0
	for {
		buf.Reset()
		gw.Reset(&buf)
		// Heal the Breach, following the paper:
		// https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9754554.
		// Essentially, we set the filename in the gzip header to a string
		// of random length.
		l, err := randomLength()
		if err != nil {
			panic(err)
		}
		gw.Header.Name = htbFilename[:l]
		gw.Write([]byte(payload))
		gw.Close()
		if buf.Len() < minSeen {
			minSeen = buf.Len()
		}
		if buf.Len() > maxSeen {
			maxSeen = buf.Len()
		}
		tries++
		if maxSeen-minSeen == maxDelta {
			got := minSeen - 2
			if want != got {
				fmt.Println("wow, so wrong, want:", want, "!= got:", got)
			}
			fmt.Println("Compressed size is", got, " That took", tries, "roundtrips to figure out.")
			return
		}
	}
}

// Filename for Heal the Breach. Any ASCII string will do.
const htbFilename = "" +
	"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" +
	"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

// Returns a random filename length for Heal the Breach.
// The length is uniformly drawn from [1,100]. Such lengths are accepted
// by all browsers, according to the Heal the Breach paper.
func randomLength() (int, error) {
	const maxtries = 1_000_000
	var htbBuf [1]byte
	// Rejection sampling.
	for i := 0; i < maxtries; i++ {
		n, err := rand.Reader.Read(htbBuf[:])
		if n == 0 || err != nil {
			return 0, fmt.Errorf("gzhttp: read crypto/rand: %w", err)
		}

		b := htbBuf[0]
		b >>= 1
		if b < 100 {
			return 1 + int(b), nil
		}
	}

	err := fmt.Errorf("gzhttp: no number < 100 in %d tries", maxtries)
	return 0, err
}

Output:

Compressed size is 280  That took 104 roundtrips to figure out.

Content based padding will make that impossible whenever the sensitive content is within the buffer.

[greatroar]

The HTB paper says

The size of each request will only be reliable after making several queries with the same input and computing the average size.

But determining the average isn't necessary, as your exploit shows. Did the authors just assume the attacker doesn't know the maximum padding length for the mitigation? 😮

[klauspost]

Yeah. Even so determining that the padding delta is 99 is also pretty trivial. With, say, 400 tries you can determine that pretty reliably.

You just need the delta and the min size. Here is your paper. You can call it "cure the breach" 😜

[d-z-m]

The HTB mitigation is effectively just adding noise to the channel. The aim is to make the attack impractical, by making it substantially more difficult.

On average for padding delta of 100, it takes ~150 round trips to deduce the compressed size, which corresponds to a single test of a test character in the paper. The number of requests an attacker would have to make to extract enough secret characters from the response body will be very large, and entirely detectable/blockable by an IDS/WAF capability.

package main

import (
	"bytes"
	"compress/gzip"
	"crypto/rand"
	"fmt"
	"math"
)

func main() {
	payload := `Error: msgp: wanted array of size 7; got 5 at Cache/testbucket/go3/src/runtime/mranges_test.go (msgp.ArrayError)
       5: e:\minio\minio\internal\logger\logger.go:258:logger.LogIf()
       4: e:\minio\minio\internal\logger\logonce.go:104:logger.(*logOnceType).logOnceIf()
       3: e:\minio\minio\internal\logger\logonce.go:135:logger.LogOnceIf()
       2: e:\minio\minio\cmd\data-usage-cache.go:903:cmd.(*dataUsageCache).load()
       1: e:\minio\minio\cmd\erasure.go:467:cmd.erasureObjects.nsScanner.func3()`

	const maxDelta = 99
	const attemptsToAvg = 1000
	tries := 0
	var buf bytes.Buffer
	gw := gzip.NewWriter(&buf)
	gw.Write([]byte(payload))
	gw.Close()
	want := buf.Len()
	minSeen, maxSeen := math.MaxInt32, 0
	var entries []int
	for j := 0; j < attemptsToAvg; j++ {
		minSeen, maxSeen = math.MaxInt32, 0
		tries = 0
		for {
			buf.Reset()
			gw.Reset(&buf)
			// Heal the Breach, following the paper:
			// https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9754554.
			// Essentially, we set the filename in the gzip header to a string
			// of random length.
			l, err := randomLength()
			if err != nil {
				panic(err)
			}
			gw.Header.Name = htbFilename[:l]
			gw.Write([]byte(payload))
			gw.Close()

			if buf.Len() < minSeen {
				minSeen = buf.Len()
			}
			if buf.Len() > maxSeen {
				maxSeen = buf.Len()
			}
			tries++
			if maxSeen-minSeen == maxDelta {
				got := minSeen - 2
				if want != got {
					fmt.Println("wow, so wrong, want:", want, "!= got:", got)
				}

				entries = append(entries, tries)
				//fmt.Println("Compressed size is", got, " That took", tries, "roundtrips to figure out.")
				break
			}
		}
	}
	fmt.Printf("Average over %d tries was %d\n",attemptsToAvg, avg(entries))
}

func avg(ints []int) int {
	var sum int
	for _, v := range ints {
		sum += v
	}

	return sum / len(ints)
}

// Filename for Heal the Breach. Any ASCII string will do.
const htbFilename = "" +
	"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" +
	"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

// Returns a random filename length for Heal the Breach.
// The length is uniformly drawn from [1,100]. Such lengths are accepted
// by all browsers, according to the Heal the Breach paper.
func randomLength() (int, error) {
	const maxtries = 1_000_000
	var htbBuf [1]byte
	// Rejection sampling.
	for i := 0; i < maxtries; i++ {
		n, err := rand.Reader.Read(htbBuf[:])
		if n == 0 || err != nil {
			return 0, fmt.Errorf("gzhttp: read crypto/rand: %w", err)
		}

		b := htbBuf[0]
		b >>= 1
		if b < 100 {
			return 1 + int(b), nil
		}
	}

	err := fmt.Errorf("gzhttp: no number < 100 in %d tries", maxtries)
	return 0, err
}

[greatroar]

Understood, but the paper promises a factor 50,000 increase in the number of queries for n=100, not 150. It focuses entirely on the problem of determining the mean number of bytes added, which isn't necessary if the attacker knows how the mitigation is implemented.

[d-z-m]

Understood, but the paper promises a factor 50,000 increase in the number of queries for n=100, not 150.

True, I'm not sure why their threat model doesn't include the attacker being able to deduce the HTB size/delta, and from there being able to develop a more efficient exploit.

[greatroar]

What is even stranger is that they cite the SafeDeflate paper, which starts off by dismissing the idea of padding the compressed response as ineffective...

[adriansmares]

Why is the insertion of the content type header conditional on the entropy available for the jitter ?

Responses of size smaller than jitterBuffer never seem to actually get a Content-Type header in my tests. As a note, I am quite sure that the underlying handlers are not guaranteed to call Flush, so I don't see how the Content-Type would be set in such low-size scenarios (above the minimum size, but below jitterBuffer).

[klauspost]

@adriansmares Please open an issue with a reproducer.