Replace Hoedown (unmaintained, but works)

[FSX]

Hoedown hasn't seen any commits since 2015. AFAIK it's good enough, but I'd like a maintained library as the back-end for Misaka.

There's:

• commonmark/cmark
• raphlinus/pulldown-cmark (also see Writing Python Extensions In Rust Using PyO3)
• MD4C
• And more

For the time being I don't have time for this, but I'll keep it in mind.

[kingclowndean]

Ok

…

On Feb 27, 2019 2:31 AM, "Frank" ***@***.***> wrote: Hoedown hasn't seen any commits since 2015. AFAIK it's good enough, but I'd like a maintained library as the back-end for Misaka. There's: - commonmark/cmark <https://github.com/commonmark/cmark> - raphlinus/pulldown-cmark <https://github.com/raphlinus/pulldown-cmark> (also see Writing Python Extensions In Rust Using PyO3 <https://www.benfrederickson.com/writing-python-extensions-in-rust-using-pyo3/> ) - MD4C <https://github.com/mity/md4c> - And more <https://github.com/commonmark/CommonMark/wiki/list-of-commonmark-implementations> For the time being I don't have time for this, but I'll keep it in mind. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#69>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACpx6FwibKgzMP_lDJp49tIF5OwMNqTTks5vRlBfgaJpZM4bUFGX> .

[FSX]

Working on this now.

[FSX]

Chose MD4C.

[FSX]

Track development here: https://github.com/FSX/misaka/tree/md4c-backend

[DanielG]

Is changing the backend C library not likely to change misaka's behaviour in subtle ways and just potentially introduce new bugs? i.e. is this really a good idea?

Let me explain -- I'm currently working on packaging misaka 2.* for Debian with hoedown properly seperated out as another package rather than embedded in misaka. As part of my review of hoedown I also noticed that the project is essentially dead. In light of this I also had some security concerns but I've since done some fuzzing with AFL and I have to admit I'm impressed.

After over 1.5 billion different test cases the fuzzer still hasn't found a single crash so I'm wondering if maybe this is just an instance of a project being "complete" and not really in need of maintanance :)

Let me know what you think.

[FSX]

That's true, but Hoedown is completely dead. If I want to continue using it I have to fork and maintain it. There are newer and maintained implementations like cmark and MD4C. I feel safer using a library that's maintained. I chose to switch to a new backend and accept the risk of incompatible changes and new bugs. I do plan to bump the major version number, maintain version 2 for a while, and write some disclaimers and instructions for moving to version 3.

After over 1.5 billion different test cases the fuzzer still hasn't found a single crash [snip]

That's impressive!

…

----- Original message ----- From: "Daniel Gröber (dxld)" <[email protected]> To: FSX/misaka <[email protected]> Cc: Frank <[email protected]>, Author <[email protected]> Subject: Re: [FSX/misaka] Replace Hoedown (unmaintained, but works) (#69) Date: Thursday, November 07, 2019 13:03 Is changing the backend C library not likely to change misaka's behaviour in subtle ways and just potentially introduce new bugs? i.e. is this really a good idea? Let me explain -- I'm currently working on packaging misaka 2.* for Debian with hoedown properly seperated out as another package rather than embedded in misaka. As part of my review of hoedown I also noticed that the project is essentially dead. In light of this I also had some security concerns but I've since done some fuzzing with AFL and I have to admit I'm impressed. After over 1.5 billion different test cases the fuzzer still hasn't found a single crash so I'm wondering if maybe this is just an instance of a project being "complete" and not really in need of maintanance :) Let me know what you think. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#69?email_source=notifications&email_token=AAAPWELXDC3ATAPF4EER4RTQSP7ZJA5CNFSM4G2QKGL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDMGDVY#issuecomment-551051735>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPWEJSQUPIL33TRJUMHSLQSP7ZJANCNFSM4G2QKGLQ>.

[DanielG]

I don't want to maintain hoedown by myself either really :), but just looking at how many other libraries are using it I have a feeling efforts could be pooled to keep it maintained. Personally I think the risk of switching the backend is greater than that of just continuing to maintain hoedown.

I can totally relate to the uneasy feeling of using an unmaintained library, but are there any issues that really need fixing at the moment? Also have you talked to any other users of the library about this? Probably should get some feedback on how the people using the library feel about this.

[FSX]

There's also a specification for markdown now, Common Markdown. Hoedown is still from before the Common Markdown specification. I think that's also a good argument to switch to something else. Before Hoedown there was Sundown. And Sundown was abandoned/unmaintained, the author went to work on Common Mark and cmark. And cmark took a while to be released. That's when people forked Sundown and made Hoedown. If I remember correctly. Maybe they all moved on? I haven't looked for and spoken to other users (yet). So I'm not sure if there are any. I have to say switching to the new backend of more in the exploring phase. I was also thinking of making a new Python package and keeping Misaka the same.