Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add module tag to elixir test cases #3178

Merged
merged 1 commit into from
Sep 30, 2020
Merged

Add module tag to elixir test cases #3178

merged 1 commit into from
Sep 30, 2020

Conversation

jiangphcn
Copy link
Contributor

@jiangphcn jiangphcn commented Sep 29, 2020

Overview

Some elixir test cases don't have actual module tag. Add tags to help include or exclude them in CI test.

Testing recommendations

make elixir

Related Issues or Pull Requests

Checklist

@dottorblaster
Copy link
Member

We already did a bit of categorization with @iilyak, :cluster and :single_node for example were meant for that purpose 😅 Maybe we need to put up a stronger naming convention?

@jiangphcn
Copy link
Contributor Author

The kind related tag is still kept. We just want to add new module tags to give fine-grained classification for elixir test cases.

Copy link
Contributor

@jjrodrig jjrodrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok for me

I've been porting some of these tests and I found difficult to define a meaningful tag that helps to categorize tests.

Copy link
Member

@dottorblaster dottorblaster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it is ok for JJ, then it's ok for me 😄

@iilyak please take a look at this and tell us what you think, maybe some sort of stronger and more fine-grained naming convention would be useful, I'm very interested in your opinion

Some elixir test cases don't have actual module tag. Add tags to
help include or exclude them in CI test.
@iilyak
Copy link
Contributor

iilyak commented Sep 29, 2020

I've noticed that most of the tags in this PR ended up to be unique ones. The only exceptions is docs which I bet used somewhere else. This means the granularity level is a single module. I am not sure it useful to do it by module. We can run tests for a single module already. We also can run tests for list of modules as well:

make elixir tests=test/elixir/test/basics_test.exs,test/elixir/test/replication_test.exs,test/elixir/test/map_test.exs,test/elixir/test/all_docs_test.exs,test/elixir/test/bulk_docs_test.exs

I think the better grouping would be based on the big features of CouchDB we intend to test. Let's say a developer is updating the view engine. In this case he/she would be interested in running all tests for view functionality. Because changes in that area might break related functionality. So I propose to look at tagging using following heuristics (just an idea and subject for discussion):

  • tag which is set on a single module is an exception not a norm
  • think about what component (or feature) of CouchDB given module tests to come up with a tag
    • we can use cover information to assist as initially but it could be an overkill
  • determine where (in which erlang application) the endpoint we test from given test module is handled and tag after application name (mango, mem3, setup, deyfus)
    find src/ -name '*_httpd_handlers.erl' | xargs grep '_handler(' | grep -v no_match |  cut -d':' -f2
    url_handler(<<"_membership">>) -> fun mem3_httpd
    url_handler(<<"_reshard">>) -> fun mem3_reshard_httpd
    db_handler(<<"_shards">>) -> fun mem3_httpd
    db_handler(<<"_sync_shards">>)   -> fun mem3_httpd
    url_handler(<<"_cluster_setup">>) -> fun setup_httpd
    db_handler(<<"_index">>)        -> fun mango_httpd
    db_handler(<<"_explain">>)      -> fun mango_httpd
    db_handler(<<"_find">>)         -> fun mango_httpd
    db_handler(<<"_index">>)        -> fun mango_httpd
    db_handler(<<"_explain">>)      -> fun mango_httpd
    db_handler(<<"_find">>)         -> fun mango_httpd
    url_handler(<<"_search_analyze">>) -> fun dreyfus_httpd
    db_handler(<<"_search_cleanup">>)  -> fun dreyfus_httpd
    design_handler(<<"_search">>)      -> fun dreyfus_httpd
    design_handler(<<"_search_info">>) -> fun dreyfus_httpd
    ...
    

@iilyak
Copy link
Contributor

iilyak commented Sep 29, 2020

JFYI: It is also possible to run individual tests within the module

make elixir tests=test/elixir/test/basics_test.exs:155

Copy link
Contributor

@iilyak iilyak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is nothing wrong with this PR. I don't mind having extra tags if folks find it useful.
We can do other round of classification as a separate PR.
+1

@jiangphcn
Copy link
Contributor Author

thanks to all of you, @dottorblaster, @jjrodrig and @iilyak. Let me merge for now and we can make further classification later.

@jiangphcn jiangphcn merged commit 1719500 into master Sep 30, 2020
@jiangphcn jiangphcn deleted the add-tag-to-elixir branch September 30, 2020 15:04
jjrodrig pushed a commit to jjrodrig/couchdb that referenced this pull request Oct 7, 2020
Some elixir test cases don't have actual module tag. Add tags to
help include or exclude them in CI test.
jjrodrig pushed a commit that referenced this pull request Oct 7, 2020
Some elixir test cases don't have actual module tag. Add tags to
help include or exclude them in CI test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants