Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch login theme to keycloak v2 #30319

Draft
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

hmlnarik
Copy link
Contributor

@hmlnarik hmlnarik commented Jun 11, 2024

This PR upgrades the keycloak.v2 login theme from experimental to default and fixes few remaining bugs and testsuites.

EDIT: The themes were fixed in #30656

Notes:

  • Most of the changes are related to the testsuite, namely updating the selectors
  • The tests are now run in a proper browser as this is needed for the keycloak.v2 login theme
    • Several tests (FIPS, SSSD) are run in containers which do not have a browser. These still run in legacy login theme (base or keycloak). For keycloak theme to be removed, these tests should depend only on a base theme or be run with a browser. This can be done as a follow-up

Fixes: #29009

Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch 5 times, most recently from f27f246 to 082f0d2 Compare June 11, 2024 12:03
Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest#claimIsPropagatedOnAllLoginsWhenNameMatchesAndSyncModeIsForce

Keycloak CI - Store IT

java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsPropagatedOnAllLoginsWhenNameMatchesAndSyncModeIsForce(OidcClaimToUserSessionNoteMapperTest.java:109)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...

Report flaky test

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch from 173151e to 6bee030 Compare June 11, 2024 16:15
Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest#claimIsPropagatedOnFirstLoginOnlyWhenNameMatchesAndSyncModeIsImport

Keycloak CI - Store IT

java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsPropagatedOnFirstLoginOnlyWhenNameMatchesAndSyncModeIsImport(OidcClaimToUserSessionNoteMapperTest.java:98)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...
java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsPropagatedOnFirstLoginOnlyWhenNameMatchesAndSyncModeIsImport(OidcClaimToUserSessionNoteMapperTest.java:98)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...

Report flaky test

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest#claimIsPropagatedOnAllLoginsWhenNameMatchesAndSyncModeIsForce

Keycloak CI - Store IT

java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsPropagatedOnAllLoginsWhenNameMatchesAndSyncModeIsForce(OidcClaimToUserSessionNoteMapperTest.java:119)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...

Report flaky test

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch from 2d55a1f to 718c817 Compare June 12, 2024 07:48
Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1021-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch from 718c817 to dea8f2c Compare June 12, 2024 09:14
hmlnarik added a commit that referenced this pull request Jun 21, 2024
This fixes the findings from #30319.

The tests are not updated and are run on legacy keycloak theme.
The new login theme is kept disabled, since otherwise it would
be treated as default and used by tests which is said to be
unacceptable due to test duration.

Fixes: #29009

Signed-off-by: Hynek Mlnarik <[email protected]>
@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch from f989f43 to fe5fa8d Compare June 21, 2024 13:23
@hmlnarik hmlnarik marked this pull request as draft June 21, 2024 13:24
Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1022-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

org.keycloak.testsuite.account.AccountRestServiceTest#listApplicationsWithoutPermission

Keycloak CI - Java Distribution IT (windows-latest - temurin - 17)

java.lang.RuntimeException: Could not create statement
	at org.jboss.arquillian.junit.Arquillian.methodBlock(Arquillian.java:307)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
...

Report flaky test

Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
alert-error -> pf-v5-c-alert
XPath for profile inputs

Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
The `base` theme can be used for SSSD tests.
Several tests from the FIPS testsuite look for a feedback message
which in turn is only available in the `keycloak` theme. Thus the
FIPS testsuite currently defers to `keycloak` theme. Ideally, it
would also use the `base` theme only.

Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
Signed-off-by: Hynek Mlnarik <[email protected]>
@hmlnarik hmlnarik force-pushed the 29009-Switch-login-theme-to-keycloak-v2-Stabilize-testsuite-after-the-switch-main branch from fe5fa8d to c911eaf Compare July 9, 2024 15:51
Copy link

@keycloak-github-bot keycloak-github-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
Copy link

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest#claimIsNotPropagatedWhenNameDoesNotMatch

Keycloak CI - Store IT (pus-rc)

java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsNotPropagatedWhenNameDoesNotMatch(OidcClaimToUserSessionNoteMapperTest.java:128)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...

Report flaky test

org.keycloak.testsuite.adapter.servlet.SAMLServletAdapterTest#multiTenant1SamlTest

Keycloak CI - Base IT

org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for wrapped: text ('principal=user-tenant1') to be present in text in element found by By.xpath: //body (tried for 5 second(s) with 500 milliseconds interval)
Build info: version: '4.21.0', revision: '79ed462ef4'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.5.0-1022-azure', java.version: '21.0.3'
Driver info: org.openqa.selenium.chrome.ChromeDriver_ByGraphene
...

Report flaky test

org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest#claimIsPropagatedOnFirstLoginOnlyWhenNameMatchesAndSyncModeIsImport

Keycloak CI - Store IT (pus-ec)

java.lang.RuntimeException: Failed to deserialize token
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.toAccessToken(OidcClaimToUserSessionNoteMapperTest.java:154)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.login(OidcClaimToUserSessionNoteMapperTest.java:145)
	at org.keycloak.testsuite.broker.OidcClaimToUserSessionNoteMapperTest.claimIsPropagatedOnFirstLoginOnlyWhenNameMatchesAndSyncModeIsImport(OidcClaimToUserSessionNoteMapperTest.java:98)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...

Report flaky test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch login theme to keycloak.v2: Stabilize testsuite after the switch
6 participants