From f33a629086618e95dde0ebcdbf4238232a55bf94 Mon Sep 17 00:00:00 2001
From: "Neely, Travis M" <travis.neely@att.com>
Date: Mon, 19 Apr 2021 19:57:49 +0000
Subject: [PATCH] Fix race condition for grastate.dat

There seems to be a race condition involving the grastate.dat file.
Upon creation of a new mariad-server pod the file would exist however,
it is not populated for a short period of time. It seems to take
around 15-20 seconds for this file to be populated. However there is
a separate thread which is attempting to read the file and tends to
end in an IndexError exception killing the thread which maintains the
grastate.dat file until the pod is restarted. This patchset adds a
loop to check for up to 60 seconds for the file to be populated
before attempting to continue, thus giving the file time to be
populated.

Change-Id: I2f2a801aa4528a7af61797419422572be1c82e75
---
 mariadb/Chart.yaml                  |  2 +-
 mariadb/templates/bin/_start.py.tpl | 15 +++++++++++++--
 releasenotes/notes/mariadb.yaml     |  1 +
 3 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/mariadb/Chart.yaml b/mariadb/Chart.yaml
index 1b632a6fe..55b5f28b5 100644
--- a/mariadb/Chart.yaml
+++ b/mariadb/Chart.yaml
@@ -15,7 +15,7 @@ apiVersion: v1
 appVersion: v10.2.31
 description: OpenStack-Helm MariaDB
 name: mariadb
-version: 0.1.12
+version: 0.1.13
 home: https://mariadb.com/kb/en/
 icon: http://badges.mariadb.org/mariadb-badge-180x60.png
 sources:
diff --git a/mariadb/templates/bin/_start.py.tpl b/mariadb/templates/bin/_start.py.tpl
index 53de4c4ac..d86d546bb 100644
--- a/mariadb/templates/bin/_start.py.tpl
+++ b/mariadb/templates/bin/_start.py.tpl
@@ -497,8 +497,19 @@ def get_grastate_val(key):
     """
     logger.debug("Reading grastate.dat key={0}".format(key))
     try:
-        with open("/var/lib/mysql/grastate.dat", "r") as myfile:
-            grastate_raw = [s.strip() for s in myfile.readlines()]
+        # This attempts to address a potential race condition with the initial
+        # creation of the grastate.date file where the file would exist
+        # however, it is not immediately populated. Testing indicated it could
+        # take 15-20 seconds for the file to be populated. So loop and keep
+        # checking up to 60 seconds. If it still isn't populated afterwards,
+        # the IndexError will still occur as we are seeing now without the loop.
+        time_end = time.time() + 60
+        while time.time() < time_end:
+            with open("/var/lib/mysql/grastate.dat", "r") as myfile:
+                grastate_raw = [s.strip() for s in myfile.readlines()]
+            if grastate_raw:
+                break
+            time.sleep(1)
         return [i for i in grastate_raw
                 if i.startswith("{0}:".format(key))][0].split(':')[1].strip()
     except IndexError:
diff --git a/releasenotes/notes/mariadb.yaml b/releasenotes/notes/mariadb.yaml
index 39e049e94..6eede479e 100644
--- a/releasenotes/notes/mariadb.yaml
+++ b/releasenotes/notes/mariadb.yaml
@@ -13,4 +13,5 @@ mariadb:
   - 0.1.10 Rename mariadb backup identities
   - 0.1.11 Disable mariadb mysql history client logging
   - 0.1.12 Set strict permission on mariadb data dir
+  - 0.1.13 Fix race condition for grastate.dat
 ...