Clay should come with more clay out the box

[dmg46664]

It would be great if clay came with more opinions out the box, like in the examples it would be great if it had a date example.

First thing I did to clean up my dataset was 30 minutes working out:

;;(local-date "15/09/2018 05:00")
(defn local-date [input]
  (try (ld/parse
    input
    (DateTimeFormatter/ofPattern "dd/MM/yyyy HH:mm"))
       (catch DateTimeParseException e
         nil)
       ))

(->> "data.csv"
     slurp
     csv/read-csv
     clerk/use-headers
;;specter
     (sp/transform [:rows sp/ALL sp/FIRST] local-date)
     )

I had previously read https://nextjournal.com/schmudde/java-time
but there was no format example there (edit: there was, just missed it on scan), so copy that in and work out
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html

Couldn't find any date examples searching in Clerk and Clerk demos code.

I would much rather have looked at some similar examples in clay and copy and pasted to solve my problem.

Although I would prefer it all bundled in clay (one dependency), I don't mind if it is separated, as long as it's easy to plonk in without much reading. I don't do data analysis enough, so that every time I make a new notebook, it's usually some such similar chore.

Perhaps there is a better way... using techml perhaps? But I'll reach for what's available, so this ticket suggests making more clay easily available.

[dmg46664]

I know legitimately this issue could just as easily be lodged with Clerk, but if clay is written as the glue layer that connects everything, it makes sense that for adoption it is here 😄

[daslu]

Thanks, @dmg46664. That is very helpful.

Indeed tech.ml.dataset (also conveniently wrapped by Tablecloth) makes it a bit easier to handle such date-time situations.

With Tablecloth, given a file data.csv

x
13/05/2022 14:49
13/05/2022 14:50

you could conveniently create a dataset with a datetime column as follows:

(require [tablecloth.api :as tc])

(-> "/tmp/data.csv"
    (tc/dataset
     {:parser-fn {"x" [:local-date-time "dd/MM/yyyy HH:mm"]}}))

or, if you have to have keyword column names:

(-> "/tmp/data.csv"
    (tc/dataset
     {:key-fn keyword
      :parser-fn {:x [:local-date-time "dd/MM/yyyy HH:mm"]}}))

This will render nicely in Clay if you include extension/dataset when starting it.

Is it making sense for your use case?

---

I think you are so right, that a cookbook for such examples is very much needed. There is a new cookbook project initiated by @kiramclean that was discussed in the recent visual-tools meeting, and I am very hopeful about that.

Past attempts at creating cookbooks have taught us that it would not be easy to create something that would grow well and stay clear and up-to-date through time. Indeed Clay is an attempt to learn from such past experiences, at least on the tooling aspect of it.

[dmg46664]

Thanks for the reply.
Turns out that echarts expects a date string, so the usefulness of the above exercise was limited practically... except that it does indicate the usefulness of a fast-forward guide to get up to speed quickly.

Thanks for the tablecloth tip.

[dmg46664]

@daslu Tablecloth is really excellent thanks! 🙏
Solved a load of other issues, and deprecated some other loading code of mine.