You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In MongoDB as well as Solr Services, we use the take() method on the BlockingQueue queue,
waiting for incoming records. This may lead to some records not flushed into the sinks.
Consider the following scenario:
In a short period of time lower than the flush interval (and at the very startup of the service or just after a batch has been flushed) a bulkput processor sends few records but less than the batch size to the service.
If there are no more immediate incoming records, and as we use the blocking method take(), those few records are blocked (in the prepared query waiting for processing) until some new records arrive.
Proposed fix: use the poll() method which has timeout support (as done for instance in the Chronix service)
In MongoDB as well as Solr Services, we use the take() method on the BlockingQueue queue,
waiting for incoming records. This may lead to some records not flushed into the sinks.
Consider the following scenario:
In a short period of time lower than the flush interval (and at the very startup of the service or just after a batch has been flushed) a bulkput processor sends few records but less than the batch size to the service.
If there are no more immediate incoming records, and as we use the blocking method take(), those few records are blocked (in the prepared query waiting for processing) until some new records arrive.
Proposed fix: use the poll() method which has timeout support (as done for instance in the Chronix service)
MongoDB:
https://github.com/Hurence/logisland/blob/develop/logisland-services/logisland-mongodb/logisland-mongodb-client-service/src/main/java/com/hurence/logisland/service/mongodb/MongoDBUpdater.java#L81
Solr:
https://github.com/Hurence/logisland/blob/develop/logisland-services/logisland-solr-client-service/logisland-solr-client-service-api/src/main/java/com/hurence/logisland/service/solr/api/SolrUpdater.java#L52
Chronix:
https://github.com/Hurence/logisland/blob/develop/logisland-services/logisland-solr-client-service/logisland-solr_6_4_2-chronix-client-service/src/main/java/com/hurence/logisland/service/solr/ChronixUpdater.java#L82
The text was updated successfully, but these errors were encountered: