Use higher QPS for secrets reencryption

[dereknola]

Proposed Changes

• Use a separate client go for updating secrets, the default wrangler values strangle the throughput of the secrets encryption controller.

Types of Changes

Feature Enhancement

Verification

Time to reencrypt 1000 "basic" secrets (secrets are simple strings)

echo "this is a file" > file.txt && for i in {1..1000}; do echo test$i >> file.txt; kubectl create secret generic test$i --from-file=file.txt; done

Master	PR
207s	4s

Time to reencrypt 200 "large" secrets (1000K size, aka the maximum secret size)

head -c 1000K </dev/urandom  > file.txt && for i in {1..200}; do echo test$i >> file.txt; kubectl create secret generic test$i --from-file=file.txt; done

Master	PR
42s	17s

Testing

Already Covered

Linked Issues

#10581

User-Facing Change

Further Comments

This dramatically cuts the time for secrets encryption. It is possible we no longer even need a controller for this task, as even with 10K+ secrets, the timeline is now seconds not tens of minutes.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.

Project coverage is 43.56%. Comparing base (d4c3422) to head (9615189).
Report is 1 commits behind head on master.

Files	Patch %	Lines
pkg/secretsencrypt/controller.go	0.00%	6 Missing ⚠️
pkg/daemons/control/server.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10571      +/-   ##
==========================================
- Coverage   49.37%   43.56%   -5.82%     
==========================================
  Files         179      179              
  Lines       14950    14951       +1     
==========================================
- Hits         7382     6513     -869     
- Misses       6207     7244    +1037     
+ Partials     1361     1194     -167     

Flag	Coverage Δ
e2etests	36.34% <0.00%> (-9.75%)	⬇️
inttests	36.74% <0.00%> (+0.04%)	⬆️
unittests	13.39% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.