"instructor solution cells" (removed upon assignment)

[haraldschilly]

If what I below doesn't exist, it's an enhancement idea.

An instructor asked about having type of cell, say "instructor cells", which is used to check if the tasks work, or to check if several variations are also ok. They're more than just those code snippets in the answers, which are stripped away. They could also be pure text, containing explanations for colleagues and not for students.

Regarding functionality, those cells are simply removed when the notebook is assigned.

[perllaghu]

This partially exists, and partially requires good coding on the part of the instructor.

1. python has assert a == b so a series of asserts combined with boolean logic will give you the variation
2. There are hidden tests which are not displayed in the released document, but used in grading, so the liberal use of comments will help there.
3. Also note there's the === BEGIN MARK SCHEME === option as well.

See this page for more details: https://nbgrader.readthedocs.io/en/latest/user_guide/creating_and_grading_assignments.html

[williamstein]

We've decided that definitely the workarounds @perllaghu mention above are too confusing and not sufficient for our users. We are thus going to come up with some extension to the nbgrader format that allows for cells that get removed in the student version. It would be nice if we could just solve this problem in a 100% nbgrader compatible way, but in the meantime we have to solve it somehow, and will try to do so in a minimally incompatible way...

[williamstein]

Can you clarify what is meant by

partially requires good coding on the part of the instructor.

Does "good coding" mean "using hacks" or does it literally mean "good quality coding and doing things right"? I can't really tell. Sorry that I'm still pretty naive about nbgrader, and I appreciate all that you've done. I'm just a little confused by the English above.

[choldgraf]

@williamstein as you're exploring different syntax etc for grading/removing cells/etc, feel free to connect on Jupyter Book as well. We may need to support similar kinds of things (I'm not sure exactly what yet), and also have some cell tags etc used to mark "remove this cell". If it's helpful to standardize on those (or to define some new metadata standard that doesn't exist yet), let me know and I can try to point in the right direction.

[perllaghu]

Can you clarify what is meant by

partially requires good coding on the part of the instructor.

Does "good coding" mean "using hacks" or does it literally mean "good quality coding and doing things right"? I can't really tell. Sorry that I'm still pretty naive about nbgrader, and I appreciate all that you've done. I'm just a little confused by the English above.

It means "writing well thought out questions, and tests for the values they return"

I'll give you a fairly simple example:

The test the student sees is:

A lazy student can write:

def squares(n):
  if n == 0 or n == -4:
    raise ValueError
  if n == 1:
    return [1]
  if n == 10:
    return [1, 4, 9, 16, 15, 36, 49, 64, 81, 100]

hidden tests will capture this.

Having said that - making nbgrader is always good....

[williamstein]

Thanks. I've gone ahead and implemented a new cell type "Instructor only" in CoCalc which lets a user create any number of cells with any content at all, and they are all completely removed from the student version of the notebook. It really does seem like an extremely good idea for a feature. I implemented by adding a new feild to the nbgrader metadata called "remove". If true, then the cell is removed during the processing step. I never store it if it is false, in order to avoid breaking interop with official nbgrader if users don't use this feature.

Some screenshots are here: sagemathinc/cocalc#4706 (comment)

I of course hope that somebody agrees that this is a good idea and implements this for official nbgrader with the exact same metadata format....

[psychemedia]

Re: new cell types: cell tags provide a simple way of allowing notebook cell customisation.

In a classic notebook, the UI handlers can iterate through cells and render them/style them/process them based on tag, nbconvert etc can handle tagged cells through custom stylesheets, and other simple tools can process notebook .ipynb files with reference to cell tags. It's easy enough to add tags manually to cells or create toolbar buttons that toggle tags associated with cells.

[haraldschilly]

cell tags

just to understand this correctly: if a student opens the notebook in a slightly different way, i.e. without the necessary extension, all solution cells are visible? The point about stripping these cells is to make sure that students can't access them.

[psychemedia]

@haraldschilly Without an extension to process the tagged cell in a notebook, the cell just appears in the normal way. Offline, it's easy enough to create a tool that will process a tagged notebook to strip out tagged cells, apply stylng in a "hard" way, etc

[perllaghu]

cell tags

just to understand this correctly: if a student opens the notebook in a slightly different way, i.e. without the necessary extension, all solution cells are visible? The point about stripping these cells is to make sure that students can't access them.

To clarify..... the nbgrader generate process physically changes the notebook, removing text from the file and inserting other text - this is what the '### BEGIN SOLUTION, === HIDDEN TESTS, etc mark.

The autograde process then copies some of those items back into the notebook, before running the notebook.

.... so nbgrader does materially strip content from cells.

@psychemedia 's point is that for other extensions, you are changing the display of the file, not changing the actual file.

[nthiery]

+1 to having an "instructor only" cell type. I independently came to this solution in my
workflow, using a cell tag for now, which is not super convenient.

[williamstein]

To follow up on this, I did fully implement this in CoCalc:

This shows what I did with nbgrader metadata to encode this:

Basically, I added a new boolean called "remove" -- when true, that cell gets removed from the student version.