Skip to content

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang.

License

Notifications You must be signed in to change notification settings

devchat-ai/gopool

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GoPool

Welcome to GoPool, a project where 95% of the code is generated by GPT. You can find the corresponding list of Commit and Prompt at pro.devchat.ai.

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang. It is designed to manage and recycle a pool of goroutines to complete tasks concurrently, improving the efficiency and performance of your applications.

Features

  • Task Queue: GoPool uses a thread-safe task queue to store tasks waiting to be processed. Multiple workers can simultaneously fetch tasks from this queue.

  • Dynamic Worker Adjustment: GoPool can dynamically adjust the number of workers based on the number of tasks and system load.

  • Graceful Shutdown: GoPool can shut down gracefully. It stops accepting new tasks and waits for all ongoing tasks to complete before shutting down when there are no more tasks or a shutdown signal is received.

  • Error Handling: GoPool can handle errors that occur during task execution. For example, it can provide an error callback function.

  • Task Timeout Handling: GoPool can handle task execution timeouts. For example, it can set a timeout period, and if a task is not completed within this period, the task is considered failed.

  • Task Priority: GoPool supports task priority. Tasks with higher priority are processed first.

  • Task Result Retrieval: GoPool provides a way to retrieve task results. For example, it can provide a result callback function.

  • Task Retry: GoPool provides a retry mechanism for failed tasks. For example, it can set the number of retries and the retry interval.

  • Task Progress Tracking: GoPool provides task progress tracking. For example, it can provide a progress callback function or a method to query the current task progress.

  • Concurrency Control: GoPool can control the number of concurrent tasks to prevent system overload.

  • Lock Customization: GoPool supports different types of locks. You can use the built-in sync.Mutex or a custom lock such as spinlock.SpinLock.

Installation

To install GoPool, use go get:

go get -u github.com/devchat-ai/gopool

Usage

Here is a simple example of how to use GoPool with sync.Mutex:

package main

import (
    "sync"
    "time"

    "github.com/devchat-ai/gopool"
)

func main() {
    pool := gopool.NewGoPool(100)
    for i := 0; i < 1000; i++ {
        pool.AddTask(func() {
            time.Sleep(10 * time.Millisecond)
        })
    }
    pool.Release()
}

And here is how to use GoPool with spinlock.SpinLock:

package main

import (
    "time"

    "github.com/daniel-hutao/spinlock"
    "github.com/devchat-ai/gopool"
)

func main() {
    pool := gopool.NewGoPool(100, gopool.WithLock(new(spinlock.SpinLock)))
    for i := 0; i < 1000; i++ {
        pool.AddTask(func() {
            time.Sleep(10 * time.Millisecond)
        })
    }
    pool.Release()
}

Performance Testing

We have conducted several performance tests to evaluate the efficiency and performance of GoPool. Here are the results:

  • TestGoPoolWithMutex:
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithMutex$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN   BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex
BenchmarkGoPoolWithMutex-10            2         803105167 ns/op        17416408 B/op    1017209 allocs/op
PASS
ok      github.com/devchat-ai/gopool    2.586s
  • TestGoPoolWithSpinLock:
$ go test -benchmem -run=^$ -bench ^BenchmarkGoPoolWithSpinLock$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN   BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock
BenchmarkGoPoolWithSpinLock-10                 2         662952562 ns/op        17327176 B/op    1016087 allocs/op
PASS
ok      github.com/devchat-ai/gopool    2.322s
  • BenchmarkGoroutines:
$ go test -benchmem -run=^$ -bench ^BenchmarkGoroutines$ github.com/devchat-ai/gopool

goos: darwin
goarch: arm64
pkg: github.com/devchat-ai/gopool
=== RUN   BenchmarkGoroutines
BenchmarkGoroutines
BenchmarkGoroutines-10                 3         371622847 ns/op        96642458 B/op    2005219 allocs/op
PASS
ok      github.com/devchat-ai/gopool    2.410s

Please note that the actual performance may vary depending on the specific use case and system environment.

About

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages