Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix heisenbug #5027

Merged
merged 7 commits into from
Jan 18, 2021
Merged

Fix heisenbug #5027

merged 7 commits into from
Jan 18, 2021

Conversation

style95
Copy link
Member

@style95 style95 commented Nov 17, 2020

This is to fix the intermittent test failures.

Description

Intermittent test failures due to heisenbugs increase the CI time and make authors run tests multiple times.

Related issue and scope

My changes affect the following components

  • API
  • Controller
  • Message Bus (e.g., Kafka)
  • Loadbalancer
  • Invoker
  • Intrinsic actions (e.g., sequences, conductors)
  • Data stores (e.g., CouchDB)
  • Tests
  • Deployment
  • CLI
  • General tooling
  • Documentation

Types of changes

  • Bug fix (generally a non-breaking change which closes an issue).
  • Enhancement or new feature (adds new functionality).
  • Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

  • I signed an Apache CLA.
  • I reviewed the style guides and followed the recommendations (Travis CI will check :).
  • I added tests to cover my changes.
  • My changes require further changes to the documentation.
  • I updated the documentation where necessary.

@style95 style95 added the wip label Nov 17, 2020
@codecov-io
Copy link

codecov-io commented Nov 19, 2020

Codecov Report

Merging #5027 (8f8d40f) into master (a202538) will decrease coverage by 6.66%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5027      +/-   ##
==========================================
- Coverage   83.52%   76.85%   -6.67%     
==========================================
  Files         195      202       +7     
  Lines        9553     9818     +265     
  Branches      413      416       +3     
==========================================
- Hits         7979     7546     -433     
- Misses       1574     2272     +698     
Impacted Files Coverage Δ
...core/database/cosmosdb/RxObservableImplicits.scala 0.00% <0.00%> (-100.00%) ⬇️
...ore/database/cosmosdb/cache/CacheInvalidator.scala 0.00% <0.00%> (-100.00%) ⬇️
...e/database/cosmosdb/cache/ChangeFeedConsumer.scala 0.00% <0.00%> (-100.00%) ⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala 0.00% <0.00%> (-95.85%) ⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala 0.00% <0.00%> (-93.90%) ⬇️
...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala 0.00% <0.00%> (-92.31%) ⬇️
...enwhisk/connector/kafka/KamonMetricsReporter.scala 0.00% <0.00%> (-83.34%) ⬇️
...e/database/cosmosdb/cache/KafkaEventProducer.scala 0.00% <0.00%> (-78.58%) ⬇️
...whisk/core/database/cosmosdb/CosmosDBSupport.scala 0.00% <0.00%> (-74.08%) ⬇️
...ore/database/azblob/AzureBlobAttachmentStore.scala 11.53% <0.00%> (-60.58%) ⬇️
... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a202538...8f8d40f. Read the comment docs.

@@ -435,7 +418,7 @@ class ActionLimitsTests extends TestHelpers with WskTestHelpers with WskActorSys
}

for (a <- 1 to 10) {
val run = wsk.action.invoke(name, Map("payload" -> "128".toJson))
val run = wsk.action.invoke(name, Map("payload" -> "128".toJson), blocking = true)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is supposed to reuse a container, so I change them to blocking calls.

@style95 style95 changed the title [WIP] Fix heisenbug Fix heisenbug Nov 23, 2020
@style95 style95 removed the wip label Nov 23, 2020
@style95
Copy link
Member Author

style95 commented Nov 23, 2020

I would re-run all tests multiple times and merge this if there is no further issue.

@style95
Copy link
Member Author

style95 commented Nov 23, 2020

Handling

org.apache.openwhisk.core.limits.ConcurrencyTests > Action concurrency limits should execute activations sequentially when concurrency = 1 STANDARD_ERROR
    org.scalatest.exceptions.TestFailedException: 502 was not equal to 200
    	at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:343)
    	at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6919)
    	at common.rest.RunRestCmd.validateStatusCode(WskRestOperations.scala:1165)
    	at common.rest.RunRestCmd.validateStatusCode$(WskRestOperations.scala:1162)
    	at common.rest.RestActionOperations.validateStatusCode(WskRestOperations.scala:244)
    	at common.rest.RunRestCmd.invokeAction(WskRestOperations.scala:1358)
    	at common.rest.RunRestCmd.invokeAction$(WskRestOperations.scala:1331)
    	at common.rest.RestActionOperations.invoke(WskRestOperations.scala:388)
    	at org.apache.openwhisk.core.limits.ConcurrencyTests.$anonfun$new$21(ConcurrencyTests.scala:137)
    	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
    	at scala.util.Success.$anonfun$map$1(Try.scala:255)
    	at scala.util.Success.map(Try.scala:213)
    	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
    	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
    	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    	at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
    	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
    	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
    	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
    	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
    	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
org.apache.openwhisk.core.limits.ConcurrencyTests > Action concurrency limits should execute activations sequentially when concurrency = 1 FAILED
    org.scalatest.exceptions.TestFailedException: 502 was not equal to 200
        at org.scalatest.MatchersHelper$.indicateFailure(MatchersHelper.scala:343)
        at org.scalatest.Matchers$AnyShouldWrapper.shouldBe(Matchers.scala:6919)
        at common.rest.RunRestCmd.validateStatusCode(WskRestOperations.scala:1165)
        at common.rest.RunRestCmd.validateStatusCode$(WskRestOperations.scala:1162)
        at common.rest.RestActionOperations.validateStatusCode(WskRestOperations.scala:244)
        at common.rest.RunRestCmd.invokeAction(WskRestOperations.scala:1358)
        at common.rest.RunRestCmd.invokeAction$(WskRestOperations.scala:1331)
        at common.rest.RestActionOperations.invoke(WskRestOperations.scala:388)
        at org.apache.openwhisk.core.limits.ConcurrencyTests.$anonfun$new$21(ConcurrencyTests.scala:137)

@style95 style95 changed the title Fix heisenbug [WIP] Fix heisenbug Dec 1, 2020
@style95 style95 changed the title [WIP] Fix heisenbug Fix heisenbug Jan 18, 2021
@style95 style95 merged commit 2d0c8a7 into apache:master Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants