LibSQL: Persisted tables with a large number of columns corrupt the database

[trflynn89]

We seem to have invalid data from a persisted DB when it contains a table with a "large" (~12) number of columns. For example, trying to DESCRIBE that table, the private master.internal_describe_table used for DESCRIBE should have 2 columns itself, but has 0 (m_data.size() in the below assertion is 0).

21.069 TestSqlStatementExecution(46:46): ASSERTION FAILED: ix < m_data.size()
./Userland/Libraries/LibSQL/Tuple.cpp:102
21.069 [#0 TestSqlStatementExecution(46:46)]: Terminating TestSqlStatementExecution(46) due to signal 6
21.073 [#0 Finalizer Task(5:5)]: Backtrace:
0x00000000deadc0de  Kernel::Processor::switch_context(Kernel::Thread*&, Kernel::Thread*&) + 0x383
21.073 [#0 Finalizer Task(5:5)]: Generating coredump for pid: 46
21.085 CrashDaemon(26:26): New coredump file: /tmp/coredump/TestSqlStatementExecution_46_1667060057
21.193 CrashReporter(47:47): Started thread "", tid = 48
21.749 CrashReporter(47:48): Generating backtrace took 548 ms
21.749 CrashReporter(47:48): --- Backtrace for thread #0 (TID 46) ---
21.749 CrashReporter(47:48): 0x0000000e2376402b: [/usr/lib/libsystem.so] syscall2 +0xb (syscall.cpp:25 => syscall.cpp:24)
21.749 CrashReporter(47:48): 0x00000004ede1a0e8: [/usr/lib/libc.so] abort +0x26 (stdlib.cpp:387)
21.749 CrashReporter(47:48): 0x00000004ede2044e: [/usr/lib/libc.so] __assertion_failed.localalias +0x7e (assert.cpp:33)
21.749 CrashReporter(47:48): 0x0000000e18b3df0f: [/usr/lib/libsql.so.serenity] SQL::Tuple::operator[](unsigned long) +0x2f (Tuple.cpp:102)
21.749 CrashReporter(47:48): 0x0000000e18b03b7f: [/usr/lib/libsql.so.serenity] SQL::AST::DescribeTable::execute(SQL::AST::ExecutionContext&) const [clone .localalias] +0x30f (Describe.cpp:36)
21.753 CrashReporter(47:48): 0x0000000e18b20418: [/usr/lib/libsql.so.serenity] SQL::AST::Statement::execute(AK::NonnullRefPtr<SQL::Database>) const +0xb8 (Statement.cpp:17)
21.753 CrashReporter(47:48): 0x0000001d8c79bb6f: [/usr/Tests/LibSQL/TestSqlStatementExecution] (anonymous namespace)::try_execute(AK::NonnullRefPtr<SQL::Database>, AK::String const&) +0x319 (TestSqlStatementExecution.cpp:31)
21.753 CrashReporter(47:48): 0x0000001d8c79bcdf: [/usr/Tests/LibSQL/TestSqlStatementExecution] (anonymous namespace)::execute(AK::NonnullRefPtr<SQL::Database>, AK::String const&) +0xab (TestSqlStatementExecution.cpp:36)
21.753 CrashReporter(47:48): 0x0000001d8c7b05c6: [/usr/Tests/LibSQL/TestSqlStatementExecution] (anonymous namespace)::__test_describe_large_table_after_persist() +0x4d8 (TestSqlStatementExecution.cpp:647)
21.753 CrashReporter(47:48): 0x0000001d8c7c480f: [/usr/Tests/LibSQL/TestSqlStatementExecution] AK::Function<void ()>::CallableWrapper<void (*)()>::call() +0x15 (Function.h:151)
21.757 CrashReporter(47:48): 0x0000000ab4fc202b: [/usr/lib/libtest.so.serenity] Test::TestSuite::run(AK::NonnullRefPtrVector<Test::TestCase, 0ul> const&) [clone .localalias] +0x1ab (Function.h:91)
21.757 CrashReporter(47:48): 0x0000000ab4fc167b: [/usr/lib/libtest.so.serenity] Test::TestSuite::main(AK::String const&, int, char**) +0x42b (TestSuite.cpp:91)
21.757 CrashReporter(47:48): 0x0000001d8c79b3fe: [/usr/Tests/LibSQL/TestSqlStatementExecution] main +0xbe (TestMain.cpp:25)
21.757 CrashReporter(47:48): 0x0000001d8c79b4c8: [/usr/Tests/LibSQL/TestSqlStatementExecution] _entry +0x48 (crt0.cpp:48)

To repro, add this to Tests/LibSQL/TestSqlStatementExecution.cpp:

TEST_CASE(describe_large_table_after_persist)
{
    ScopeGuard guard([]() { unlink(db_name); });
    {
        auto database = SQL::Database::construct(db_name);
        EXPECT(!database->open().is_error());

        auto result = execute(database, "CREATE TABLE Cookies ( name TEXT, value TEXT, same_site INTEGER, creation_time INTEGER, last_access_time INTEGER, expiry_time INTEGER, domain TEXT, path TEXT, secure INTEGER, http_only INTEGER, host_only INTEGER, persistent INTEGER );");
        EXPECT_EQ(result.command(), SQL::SQLCommand::Create);
    }
    {
        auto database = SQL::Database::construct(db_name);
        EXPECT(!database->open().is_error());

        auto result = execute(database, "DESCRIBE TABLE Cookies;");
        EXPECT_EQ(result.size(), 12u);
    }
}

I've been investigating it on this branch: https://github.com/trflynn89/serenity/tree/sql_boom
EDIT: The fixes from that branch were merged in #15865.

But I'm not making much progress so maybe someone else has some ideas :)