RANGER-4932:Handling Specified key was too long; max key length error

[basapuram-kumar]

What changes were proposed in this pull request?

Proposed changes are mentioned at RANGER-4932

How was this patch tested?

tested this patch on my local cluster during the migration of ranger from 2.3.0 to 2.5.0, and with proposed changes, able to fix this issue.

[prabhjyotsingh]

LGTM!

@spolavarpau1 / @princeap173 can you please help review.

[bhavikpatel9977]

Changing the DEFAULT CHARSET for just one table might cause issue if the DEFAULT CHARSET is set to something different.

[basapuram-kumar]

Hello @bhavikpatel9977 ,
I have following approaches to discuss here.

1. Define the DEFAULT CHARSET=utf8 has been successful.
2. Reduce the size of indexed columns either with
   KEYx_trx_log_v2_trx_id (trx_id(512))) or
   KEY x_trx_log_v2_trx_id (trx_id(255)) also seems to work..

Could you please advise whether one of these approaches would be optimal, or if there are any better alternatives we should consider?
Thank you.

[basapuram-kumar]

Hi team, I'm just checking in on the status of my pull request. Would you be able to review it soon?
Would appreciate any feedback or suggestions on this PR.
Thanks!
cc: @bhavikpatel9977 , @spolavarpau1 , @princeap173 , @kumaab , @prabhjyotsingh

[basapuram-kumar]

Hi team,
I'm just checking in on the status of my pull request. Would you be able to review it soon?
Would appreciate any feedback or suggestions on this PR. Thanks

[vikaskr22]

Hi @basapuram-kumar ,
Thanks for the work !
Today I tried to understand this, and sharing my input on this.
I see merit in Bhavik's comment, changing the CHARSET at table level may create many problems. Charset changes in just one table may break JOIN operation or may result wrong output.
Sometimes, we set the Charset at Connection level in code , like we set to read using UTF-8, there it would break.

My input is, it should be defined at schema/DB level ( may be using DB server configs), means same charset for entire Ranger.

Now coming to Latin1, it will limit the characters that Ranger supports or will support in future.

On the second approach, where you suggested to reduce the length of trx_id(255), it may work. But if intention is to keep exact transactionId in the index, then it would be an issue.

Few questions:
What is the MySql version where you got this error ? What is default "storage_engine"? It should be 'InnoDb'.

I am also exploring this and will add my input here.