Streamwriter Memory Leak

[johnpharmetika]

Description

Using Streamwriter to convert large JSON arrays into XLSX (50k+ rows). At the end of the function, Streamwriter.Flush() & runtime.GC() is called, manually freeing a lot of allocated memory, but nothing from Streamwriter. Streamwriter will either hog large amounts of memory for minutes at a time, or not release it.

Steps to reproduce the issue:

	excel_file := excelize.NewFile()
	defer excel_file.Close()

	sheet_index, _ := excel_file.NewSheet("Sheet1")
	excel_file.SetActiveSheet(sheet_index)

	stream_writer, _ := excel_file.NewStreamWriter("Sheet1")

	for row := 0; row < len(json_values_array); row++ {

		active_row, _ := json_values_array[row].Array()

		interface_row := make([]interface{}, len(active_row))
		for cell := 0; cell < len(active_row); cell++ {
			interface_row[cell] = active_row[cell].Get().MarshalTo(nil)
		}

		first_cell := fmt.Sprintf("A%d", (row + 1))

		stream_writer.SetRow(first_cell, interface_row)
	}

	stream_writer.Flush()
        runtime.GC()

	excel_file.WriteTo(buffer_pointer)

Describe the results you received:
Memory leak from excelize

Describe the results you expected:
Memory released

Output of go version:

go version g01.21.1 linux/amd64

Excelize version or commit ID:

v.2.8.0

Environment details (OS, Microsoft Excel™ version, physical, etc.):
Linux kernel 5.15.0-88-generic

[xuri]

Thanks for your issue. If you print gc traces by embedding gctrace env GODEBUG=gctrace=1 go run main.go for the following code:

package main

import (
    "fmt"
    "math/rand"
    "runtime"

    "github.com/xuri/excelize/v2"
)

func main() {
    for i := 0; i < 5; i++ {
        f := excelize.NewFile()
        sw, err := f.NewStreamWriter("Sheet1")
        if err != nil {
            fmt.Println(err)
            return
        }

        for rowID := 1; rowID <= 102400; rowID++ {
            row := make([]interface{}, 50)
            for colID := 0; colID < 50; colID++ {
                row[colID] = rand.Intn(640000)
            }
            cell, err := excelize.CoordinatesToCellName(1, rowID)
            if err != nil {
                fmt.Println(err)
                break
            }
            if err := sw.SetRow(cell, row); err != nil {
                fmt.Println(err)
                break
            }
        }
        if err := sw.Flush(); err != nil {
            fmt.Println(err)
            return
        }
        if err := f.SaveAs("Book1.xlsx"); err != nil {
            fmt.Println(err)
        }
        if err := f.Close(); err != nil {
            fmt.Println(err)
        }

        fmt.Printf("[%d] GC ...\r\n", i)
        runtime.GC()
        fmt.Printf("[%d] GC Done\r\n", i)
    }
}

You can see something like this:

...
[0] GC ...
gc 15 @5.485s 0%: 0.044+0.25+0.003 ms clock, 0.53+0/0.39/0.19+0.038 ms cpu, 36->36->1 MB, 42 MB goal, 0 MB stacks, 0 MB globals, 12 P (forced)
[0] GC Done
gc 16 @5.492s 0%: 0.089+0.38+0.030 ms clock, 1.0+0.090/0.60/0.21+0.36 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 12 P
...
[1] GC ...
gc 29 @11.096s 0%: 0.038+0.20+0.002 ms clock, 0.46+0/0.44/0.20+0.031 ms cpu, 35->35->1 MB, 42 MB goal, 0 MB stacks, 0 MB globals, 12 P (forced)
[1] GC Done
gc 30 @11.103s 0%: 0.042+0.33+0.040 ms clock, 0.51+0.047/0.58/0.19+0.48 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB 
...
[2] GC ...
gc 43 @16.883s 0%: 0.13+0.72+0.005 ms clock, 1.5+0/1.4/0.081+0.068 ms cpu, 35->35->1 MB, 42 MB goal, 0 MB stacks, 0 MB globals, 12 P (forced)
[2] GC Done
gc 44 @16.894s 0%: 0.048+0.44+0.030 ms clock, 0.58+0.090/0.63/0.23+0.36 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 12 P
...

The heap size at GC starts from 35MB to 3MB at GC end on each garbage collection, the memory was not leaked. You can also observe the GC in other ways, such as go tool trace and print the memory by runtime.ReadMemStats, etc. So I've closed this issue, if you have any questions please let me know, and you can reopen this anytime.