New pagination #162

garrensmith · 2014-03-03T17:19:23Z

This is an improvement and fix on the current pagination. Currently the pagination doesn't work correctly when using startkey, endkey or any of the other query options. It also shows an incorrect value for how many documents are in the view query.

This patch is very large because pagination in Couchdb is quite complex and there are plenty of situations to cater for. This new pagination works quite differently to the previous way we had it working for a user.

A user can set the number of documents they want to view on a page. This is not related to limit options in the query options - the limit option is an overall cap of how many documents to paginate too. A limit of none is possible and is the default. If the limit option is set to 50 and a user wants 10 docs per page, the would then be able to paginate 3 pages before hitting the end.

Another change is that the api url and browser url does not change when we paginate. That happens internally and hence the new addition of urlParams and docParams. This allows Fauxton to keep track of the parameters that the user sees and the parameters that the document needs to paginate.

The majority of this work is finding the right way to paginate given a variety of query options. This is quite tricky and we had to build up scenarios of different query options and then how to paginate with them.

deathbearbrown · 2014-03-03T17:46:37Z

src/fauxton/app/addons/documents/resources.js

- this.params = options.params;
- this.skipFirstItem = false;
-
+ this.params = _.clone(options.params);


Clone is neat. Why chose this over duplicating the options.params?

I don't quite follow you here, _.clone is basically a duplicate?

NM, I googled. Garren, I can't write proper questions. Sorry

kxepal · 2014-03-03T17:51:09Z

Looks as a lot of work! But could you provide some summary what have changed, what does it improved and...why?(:

kxepal · 2014-03-03T17:52:19Z

Bug: Page sizes 50 and 100 for browsing views doesn't works. Actually, it applies last selected page size or 20.

deathbearbrown · 2014-03-03T17:53:30Z

src/fauxton/app/addons/documents/routes.js

+ // this is really ugly. But we basically need to make sure that
+ // all parameters are in the correct state and have been parsed before we
+ // calculate how to paginate the collection
+ _.each(['startkey', 'endkey', 'key'], function (key) {


can you pull this out into it's own function?

kxepal · 2014-03-03T18:00:31Z

Another moment: when paginates views, Fauxton adds limit=N argument to the URL. It's awesome! But if I set N as 11, 15, 42, 100500, Fauxton accepts the specified value, but doesn't reflect it in Per page select box.

kxepal · 2014-03-03T18:02:08Z

Also, if I set bad value for limit in URL like limit=foo, Fauxton will show Bad request error popup and resets limit to 100 while default value is 20.

deathbearbrown · 2014-03-03T18:03:00Z

src/fauxton/test/mocha/chai.js

@@ -27,10 +27,14 @@ function require(path, parent, orig) {
 // perform real require()


heh did you update chai too?

Yeah, I was adding tests and realised we needed latest chai. This is stupid to update chai.js while working on a patch. So sorry about that.

haha it's fine. I was just like "wow lotta files changed for pagination"

kxepal · 2014-03-03T18:04:44Z

Bug: Page sizes 50 and 100 for browsing views doesn't works. Actually, it applies last selected page size or 20.

More notes: it only raises if limit=N is in URL. By default it's missed, but just navigate to some document and click on "Back to _all_docs" or enter it manually.

kxepal · 2014-03-03T18:05:23Z

@deathbearbrown hm?

deathbearbrown · 2014-03-03T18:07:50Z

I think we want to move away from passing the limit parameters in the route fragment.

kxepal · 2014-03-03T18:12:39Z

@deathbearbrown if so, then Fauxton shouldn't process them. However, I see that you wanted to kill the feature: processing query params from route url allows users to exchange views requests via URL and without talks like: "open view X, click query options, set startkey=Y, endkey=Z, limit=Q, reduce=false and you'll see the result as I do"

kxepal · 2014-03-03T18:15:55Z

or another case: you configured query for specific view in Fauxton and know wanted to make the same request via curl or whatever client. Just copypaste the URL and remove /_utils/fauxton substring is better than typing all view query options manually again.

deathbearbrown · 2014-03-03T18:16:15Z

Ack I keep accidentally posting half your name.

The API URL button is what is going to show the URL for you to do curl commands, not the fauxton url.

garrensmith · 2014-03-04T09:04:57Z

@kxepal Can you explain what you mean by the 50 and 100 limits not working?

kxepal · 2014-03-04T10:16:25Z

@garrensmith get it 100% reproduce:

Navigate to any view
Open Query Options
Set Limit to lets say 20
Query the view
Collapse Query Options
Try to set page size 5 - ok
Try to set page size 20 - ok
Try to set page size 30 - it's still will be 20
Try to set page size 50 - again
Try to set page size 100 - and again
Going back to page size 10 - everything is fine

Sorry, no video: just found that gtk-recordMyDesktop generates broken picture.

kxepal · 2014-03-04T10:17:25Z

@deathbearbrown

The API URL button is what is going to show the URL for you to do curl commands, not the fauxton url.

Yes, and I was talking about sharing URL with Fauxton with all setted view query parameters. Different things.

asfbot · 2014-03-04T10:22:34Z

Simon Metson on [email protected], [email protected] replies:
That sounds like the correct behaviour. You've constrained the result set to 20 docs, so a page size >20 isn't going to show more docs. The same would happen if the DB only had 20 docs in.

kxepal · 2014-03-04T10:55:11Z

Simon:

I and @garrensmith just discussed this topic in IRC. TL;DR this is could be correct behaviour for Fauxton, but 1) it's not oblivious 2) 90% users will come to Fauxton with Futon experience and such behaviour is completely unexpected for them and could be only recognized as a bug.

If so and it's expected, Fauxton shouldn't provide options to select that doesn't works and it should explicitly declare "why functionality which I have used two minutes ago is missed". If you take a look on screen, you'll didn't notice any page size cap. Only 1) if you open Query Options, but it consumes a lot of visible space to keep it always opened 2) if you take a look on URL, but there could be a lot of parameters and @deathbearbrown going to get rid them from there.

So, whatever the behavior is, there are some problems with it.

asfbot · 2014-03-04T12:12:57Z

Simon Metson on [email protected], [email protected] replies:
I think the right thing to do is have some kind of "expecting more docs? Check your " message if the number of docs is less than the page size, and advanced options are set.

The alternative is removing the limit option, which doesn't solve the problem for other limiting options, or having conditional messaging around the page size.

deathbearbrown · 2014-03-05T19:24:41Z

Testing this now and nothing is breaking for me. +1?

garrensmith and others added 18 commits March 3, 2014 11:18

add per page view limit and new advanced options

0f69dc9

Working on doc limit

9ccfdd9

More working

fa33d4d

Set view limit on pagination

55b6ea6

fix previous paginate

2cf0f41

remove api options

69fa6c4

Paginiation with no limit

98b5472

Fix query options not toggling

4e10385

Add new pagination algorithmn

f796b38

More pagination improvements

0beb91c

More improvements

9848c76

More pagination fixes

77c06c2

Invalid combination of query options causes the dashboard to hang

747e004

Continued work

0901b9b

Improvements around parameter parsing

8a27510

More pagination goodness

cf7a92a

Fix page count on perPage change

7df61e7

Code improvements to pagination

b63c791

deathbearbrown reviewed Mar 3, 2014
View reviewed changes

Improvements from code review

7381193

garrensmith closed this Mar 10, 2014

asfgit deleted the paginate-api-options branch July 26, 2016 20:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New pagination #162

New pagination #162

garrensmith commented Mar 3, 2014

deathbearbrown Mar 3, 2014

garrensmith Mar 4, 2014

deathbearbrown Mar 5, 2014

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown Mar 3, 2014

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown Mar 3, 2014

garrensmith Mar 4, 2014

deathbearbrown Mar 5, 2014

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown commented Mar 3, 2014

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown commented Mar 3, 2014

garrensmith commented Mar 4, 2014

kxepal commented Mar 4, 2014

kxepal commented Mar 4, 2014

asfbot commented Mar 4, 2014

kxepal commented Mar 4, 2014

asfbot commented Mar 4, 2014

deathbearbrown commented Mar 5, 2014

		@@ -27,10 +27,14 @@ function require(path, parent, orig) {
		// perform real require()

New pagination #162

New pagination #162

Conversation

garrensmith commented Mar 3, 2014

deathbearbrown Mar 3, 2014

Choose a reason for hiding this comment

garrensmith Mar 4, 2014

Choose a reason for hiding this comment

deathbearbrown Mar 5, 2014

Choose a reason for hiding this comment

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown Mar 3, 2014

Choose a reason for hiding this comment

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown Mar 3, 2014

Choose a reason for hiding this comment

garrensmith Mar 4, 2014

Choose a reason for hiding this comment

deathbearbrown Mar 5, 2014

Choose a reason for hiding this comment

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown commented Mar 3, 2014

kxepal commented Mar 3, 2014

kxepal commented Mar 3, 2014

deathbearbrown commented Mar 3, 2014

garrensmith commented Mar 4, 2014

kxepal commented Mar 4, 2014

kxepal commented Mar 4, 2014

asfbot commented Mar 4, 2014

kxepal commented Mar 4, 2014

asfbot commented Mar 4, 2014

deathbearbrown commented Mar 5, 2014