./hakyll build always rebuilds everything after switching to GHC 7.8 and Hakyll 4.5.1.0 #250
Comments
|
I can confirm this. |
|
I'm looking into this, will update as I discover things. For now I think I've managed to rule out the possibility of fsnotify or one of its dependencies (e.g. hinotify) being the culprit. Hakyll does only get one event for the one file that was changed, but then it goes and updates everything. One thing I noticed is that the |
|
I also noticed that |
|
@blaenk What hakyll/GHC version are you using? Looks like a real issue. Note that something like: means that (Hakyll thinks) the underlying file has been modified. |
|
Can also reproduce with GHC 7.8.2 with following packages inside sandbox I think I could see it before GHC 7.8, so some package being at fault comes to mind. |
|
@jaspervdj can't believe I forgot to mention that! GHC 7.8.2, latest On Monday, May 5, 2014, Simonas Kazlauskas [email protected] wrote:
|
|
I agree with @nagisa that it could be a dependency, though I think not hfsnotify. The reason I believe this is because I went all the way back to bc360a3 and I still observe this behavior, unless of course the problem could have been introduced before then and it's just now conflicting with GHC 7.8.2 et al. I specifically went to that commit because it's before the global metadata series of commits, since I wondered if the recent reverts on those features were unclean and ended up causing this problem. I should note that I've noticed that all of us in this thread seem to be using a custom Pandoc compiler, i.e. not one that is |
|
This is also happening on 4.4.1.0, commit 808353b. I'm starting to think it's something related to GHC 7.8.2? |
|
@blaenk what OS are you on? What does "ghc --info" print out? |
|
@cartazio I'm on arch, with the GHC package that came out yesterday for 7.8.2. (before that was 7.6.x). Here's the output: |
|
Tired of looking into this, can't figure out any leads. Just a heads up, however. It seems the latest version of some package, can't tell which one, but I think transformers-0.4.0.0 based on this, (since mtl is already capped at < 2.0 in hakyll, which by the way should probably be relaxed since the latest version is 2.2.0.1) depends on exceptions-0.6, which breaks temporary-1.2.0.2 (pending PR #12 that aims to fix the issue). Suffice it to say everything seems to be a mess right now as packages scramble (hopefully) to fix their packages and their version constraints. I found that the only way to build hakyll at all was to specify an explicit constraint to cabal as follows: cabal install --dependencies-only --constraint 'exceptions < 0.6'That said, I still have no idea what could be causing this annoying bug. |
|
This bug completely breaks my workflow, since I use hakyll nearly every day for keeping notes. I'm willing to (and have) look into it but I'm completely at a loss as to what may be occurring. I don't have any leads to follow. The situation so far:
I'm wondering if any of you have any hunches on what the issue may be, @jaspervdj perhaps? |
|
@blaenk I'd suggest to start digging in this method: https://github.com/jaspervdj/hakyll/blob/master/src/Hakyll/Core/Provider/Internal.hs#L172 The way I'd go around it is just using |
|
I figured it out! Will post information about the bug and a fix when I've finished. |
|
Came up with the fix and submitted it as PR #251. |
It seems like System.Directory's `getModificationTime` is now providing sub-second precision if it's available in the underlying system call, which it seems to be for most systems. The problem is that when the modification time is stored, the DiffTime, the number of seconds from midnight, has its sub-second precision shaved off, effectively becoming 0 the next time it's read. When a file is then checked to see if it has been modified, the file is _always_ deemed to have been modified because this time still includes the sub-second precision, which will almost always be later in time than 0, unless of course it happened to be 0 to begin with. This simply shaves the sub-second precision off of the time returned by `getModificationTime`, and thus no longer rebuilds every single file each time any one file is saved. Closes jaspervdj#250
|
WOAH. thats a good one |
|
Impressive, great debugging there! |
Some systems can get the file modification time with sub-second granularity. However, Hakyll shaves off the sub-seconds, as defined in the Binary instance of BinaryTime, which poses a problem because when a file is checked to see if it was modified in `resourceModified`, it still contains the sub-seconds. This results in a file (almost) always being considered as having been modified. Example: 1. First go around, modification time is 3:45.325. This time is saved as 3:45.000 (i.e. sub-seconds are shaved off). 2. Second go around, modification time is again read as 3:45.325 and compared against the stored time, 3:45.000. 3:45.325 is more recent than 3:45.000, so the file is considered to have been modified. This change prevents the shaving off of sub-seconds. This will naturally work on systems that don't support sub-second granularity, as that 'field' will simply appear as all zeros. Closes jaspervdj#250
|
Thanks for pointing me in the right direction :) |
|
Released on Hackage as |
The caching doesn't seem to work properly any more with Hakyll 4.5.1.0 and GHC 7.8.2. Everything gets rebuilt every time, which makes using "./hakyll watch" harder.
Steps to reproduce (with my website code):
I tried it with older versions of my
hakyll.hsfile that definitely worked back on GHC 7.6.2, and the bug persisted. I also think I used to be on an older version of the Hakyll package itself, so it could be either a difference in the package or a difference in GHC.I haven't tried with older versions of Hakyll, but I could do that if it seems useful.
Thanks for looking at this.
The text was updated successfully, but these errors were encountered: