Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance of the QoH calculation #100

Open
12 tasks
jmiranda opened this issue Sep 14, 2015 · 2 comments
Open
12 tasks

Improve performance of the QoH calculation #100

jmiranda opened this issue Sep 14, 2015 · 2 comments
Assignees
@jmiranda
Labels
status: ready for work An issue that is ready to be assigned and started status: work in progress For issues or pull requests that are in progress but not yet completed

Comments

@jmiranda
Copy link
Member

jmiranda commented Sep 14, 2015 •
edited
Loading

It currently takes about 30 minutes to calculate and store inventory snapshot data (for a OB instance that has lots of products and locations). We should be able to improve both the query time for the calculation as well as writing records to the tables.

  • Implement inventory summary table to provide up-to-date snapshot of QoH for all items in the system
  • Use inventory summary table in places where on-demand computed QoH was being used
  • Validate inventory summary table against on-demand computed QoH
  • Validate
  • Improve performance of megamenu (outbound, inbound shipments)
  • Improve time to save records to inventory summary and inventory snapshot tables
  • Merge inventory snapshot and inventory item snapshot table
  • Add worker queue / parallel computing (Gpars) to improve performance of the QoH calculation
  • Add worker queue / parallel computing (Gpars) for populating inventory snapshot tables to compute
  • Consider using secondary caching store (redis) for QoH summary
  • Replace all inventory transactions with a correspondng adjustment transaction #165 Replace all inventory transactions with adjustment transactions
  • Allow system admin to configure which method to use when pulling aggregate QoH data (i.e. summary, snapshot, calculate).
@jmiranda jmiranda added the ready label Sep 21, 2015
@jmiranda jmiranda self-assigned this Oct 4, 2015
@jmiranda jmiranda added in progress and removed ready labels Oct 4, 2015
jmiranda added a commit that referenced this issue Oct 4, 2015
@jmiranda
Merge pull request #111 from openboxes/feature/#100-improve-performan…
3b7754c 
…ce-qoh-job

Feature/#100 improve performance of automated inventory snapshot process
jmiranda added a commit that referenced this issue Oct 4, 2015
@jmiranda
Merge pull request #112 from openboxes/feature/#100-improve-performan…
4cb9a1a 
…ce-qoh-job

fixed bug related to the table where we're saving inventory item snapshot data
@jmiranda jmiranda added this to the 0.7.4 milestone Oct 6, 2015
@jmiranda jmiranda changed the title Improve performance of the quartz job that populates the inventory snapshot table Improve performance of the quartz job that populates the inventory snapshot table Apr 7, 2016
@jmiranda jmiranda added ready and removed in progress labels Apr 7, 2016
@jmiranda jmiranda changed the title Improve performance of the quartz job that populates the inventory snapshot table Improve performance of the QoH calculation Nov 9, 2016
@jmiranda jmiranda modified the milestones: 0.7.13, 0.8.0 Nov 9, 2016
@jmiranda
Copy link
Member Author

jmiranda commented Nov 9, 2016

This one might be a hotfix so I've moved it to milestone 0.7.13 for now.

@jmiranda jmiranda mentioned this issue Nov 16, 2016
Open
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 introduced SQL script and quartz job that will be used to popula…
20f6d50 
…te inventory_item_summary table
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 improved usability of global search (now that we can retrieve qu…
b4f10f5 
…antities faster)
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 modified inventory item summary SQL script to handle some edge c…
be427d0 
…ases
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 major performance improvements for the dashboard
c7c838d
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 Added domain class so we can use GORM/HQL to query and manage th…
ac56b12 
…e current QoH values
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 added data migration changelog for inventory item summary table
0725f4a
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 fixed issue when inventory item summary data is not available yet
b505284
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 fixed requisition status list to show only valid statuses
9a061e6
jmiranda added a commit that referenced this issue Nov 16, 2016
@jmiranda
#100 dashboard / megamenu caches should be flushed when user logs out
e3e330c
@jmiranda jmiranda modified the milestones: 0.7.14, 0.7.13 Nov 16, 2016
jmiranda added a commit that referenced this issue Nov 21, 2016
@jmiranda
#100 more improvements to the dashboard widgets as well as some of th…
24df688 
…e listing pages that are linked from the dashboard
jmiranda added a commit that referenced this issue Nov 21, 2016
@jmiranda
#100 minor improvement to the look-and-feel for the purchase order fi…
c34f893 
…lter sidebar
jmiranda added a commit that referenced this issue Nov 21, 2016
@jmiranda
#100 requisition workflow and consumption report are now using QoH su…
b402d72 
…mmary table
jmiranda added a commit that referenced this issue Nov 21, 2016
@jmiranda
#100 deprecated method that was being used to calculate QoH of all pr…
13e3590 
…oducts stocked at a location in favor of summary table query
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 modified inventory item summary SQL script to handle some edge c…
bfebd30 
…ases
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 major performance improvements for the dashboard
909b597
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 Added domain class so we can use GORM/HQL to query and manage th…
02acfd9 
…e current QoH values
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 added data migration changelog for inventory item summary table
a9dfe4c
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 fixed issue when inventory item summary data is not available yet
c8160e3
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 fixed requisition status list to show only valid statuses
789264f
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 dashboard / megamenu caches should be flushed when user logs out
02c5f54
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 more improvements to the dashboard widgets as well as some of th…
5ddffe2 
…e listing pages that are linked from the dashboard
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 minor improvement to the look-and-feel for the purchase order fi…
cc25f21 
…lter sidebar
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 requisition workflow and consumption report are now using QoH su…
e6491ae 
…mmary table
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 deprecated method that was being used to calculate QoH of all pr…
16279d5 
…oducts stocked at a location in favor of summary table query
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 more dashboard cleanup
8222420
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 removed subtotals from product and expiration summary widgets to…
fc28e06 
… reduce clutter
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 Changed cron expression for the build summary tables job but it'…
b14feae 
…s still running every hour rather than every five minutes
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 dramatically improved performance of the Edit Transaction page
2160a64
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 fixed bug with cron expression used by build summary tables job …
a3824df 
…(does not fix the "every 5 minutes" vs "every hour" issue though)
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 fixed multiple bugs with the summary table build script and quer…
4022402 
…ies used to pull data from the summary table
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 changed how QoH values are queried within several components (e.…
6a4a000 
…g. fast movers, global search, and requisition pages)
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 cached all dashboard data responses by default and changed defau…
c043909 
…lt date range to "last 7 days" (from 30)
jmiranda added a commit that referenced this issue Nov 25, 2016
@jmiranda
#100 dramatically improved the rebuilding of snapshot tables by batch…
af64b3f 
…ing the insert statement
@jmiranda jmiranda removed this from the 0.7.14 milestone Dec 8, 2016
@jmiranda jmiranda added this to the 0.7.18 milestone May 6, 2017
@jmiranda jmiranda added the status: work in progress For issues or pull requests that are in progress but not yet completed label May 15, 2017
jmiranda added a commit that referenced this issue Jun 9, 2017
@jmiranda
#100 improvements to the user signup process
0f93c05
@jmiranda jmiranda removed this from the 0.7.18 milestone Aug 27, 2017
@jmiranda
Copy link
Member Author

GPars resources

@jmiranda jmiranda added status: ready for work An issue that is ready to be assigned and started and removed status:ready labels Oct 5, 2018
@vaibhavantil2 vaibhavantil2 mentioned this issue Nov 28, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jmiranda jmiranda

Labels
status: ready for work An issue that is ready to be assigned and started status: work in progress For issues or pull requests that are in progress but not yet completed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jmiranda